RoCo-ACE: Rollout-Conditioned Online Distillation for Retention-Aware Knowledge Injection RoCo-ACE：基于回放的在线蒸馏用于保留感知的知识注入

TL;DR

• RoCo-ACE introduces a rollout-conditioned online distillation objective for knowledge injection in MLLMs, addressing drift in non-updated behaviors.
• It uses reference-free/reference-conditioned likelihood contrast to reallocate distillation weight to reference-supported rollout tokens.
• ACE adds sparse reference-side anchored correction for omitted authoritative anchors without full-answer imitation.
• Across three knowledge-injection settings and six retention benchmarks, RoCo-ACE achieves the best injected-knowledge accuracy while retaining base model performance.

Why It Matters

This work is relevant because it tackles a critical challenge in updating pretrained models with new knowledge: balancing accurate knowledge injection with preserving existing capabilities. By refining online distillation techniques, RoCo-ACE offers a more precise method for adapting MLLMs to domain-specific or factual updates without catastrophic forgetting, which is vital for deploying reliable, up-to-date AI systems in dynamic environments.

Technical Details

• Rollout-Conditioned Distillation: Unlike uniform reference-conditioned distillation, RoCo-ACE dynamically adjusts supervision by comparing same-rollout reference-free and reference-conditioned likelihoods, emphasizing tokens supported by references.
• Sparse Anchored Correction (ACE): This component addresses omitted authoritative facts from rollouts by anchoring corrections on reference-side targets, avoiding reliance on full-answer imitation that may introduce noise.
• Evaluation Setup: Tested across three knowledge-injection scenarios (e.g., factual updates, domain adaptation), six retention benchmarks measuring preserved capabilities, multiple baselines (e.g., standard fine-tuning, vanilla online distillation), and various base MLLM architectures.
• Key Outcome: The method outperforms baselines in injected-knowledge accuracy while maintaining retention metrics close to the original base model, demonstrating effective balance between updating and preserving knowledge.

Industry Insight

For AI practitioners, RoCo-ACE highlights the importance of nuanced distillation strategies when updating large language models—prioritizing token-level supervision over blanket imitation can reduce drift and improve adaptability. As organizations increasingly rely on MLLMs for specialized tasks, adopting such retention-aware knowledge injection methods will be critical to ensuring models remain both current and reliable without costly retraining from scratch.

TL;DR

• 提出 RoCo-ACE 方法，通过 rollout-conditioned online distillation 解决多模态大语言模型（MLLMs）知识注入时的行为漂移问题。
• RoCo 利用同次生成的 rollout 中参考自由与参考条件的似然对比，重新分配权重以强化支持参考的 token；ACE 则引入稀疏锚点修正机制，补充权威答案中遗漏的关键事实。
• 在三种知识注入场景、六个保留基准及多种基座模型上验证，RoCo-ACE 在保持原有能力接近原始模型的同时，实现了最高的知识注入准确率。
• 该方法有效平衡了“新知识获取”与“旧有行为保留”，为 MLLM 持续学习提供了更精细的监督信号设计思路。
• 技术核心在于动态调整蒸馏损失分布，避免传统全答案拟合导致的过拟合与泛化退化。

为什么值得看

本文针对当前 MLLM 知识更新中的关键痛点——即如何在注入新事实而不破坏已有能力的情况下进行高效微调——提出了创新的在线蒸馏框架。其提出的双阶段加权策略（RoCo + ACE）不仅提升了知识吸收效率，也为后续研究提供了可复用的损失工程范式，对工业界部署实时更新的智能体具有重要指导意义。

技术解析

• Rollout-Conditioned Likelihood Contrast (RoCo)：在同一轮生成输出中，同时计算基于参考文本的条件概率和无条件概率，二者之差作为额外权重放大那些既符合参考又由模型自然生成的 token 的学习贡献，从而增强对可靠知识的聚焦。
• Anchor-Side Correction (ACE)：针对权威答案中存在但未被模型 rollout 覆盖的关键实体或关系（称为“anchor”），施加稀疏的监督信号，无需完整模仿整句回答，仅纠正缺失部分，降低干扰风险。
• Online Distillation Setup：采用自监督方式，让模型先自行生成候选响应，再根据外部知识库评估并调整内部参数，整个过程无需人工标注新数据流，适合动态环境下的增量学习。
• Evaluation Metrics & Benchmarks：使用六项 retention benchmark 衡量模型在未训练任务上的性能衰减情况，并结合 accuracy@K、F1-score 等指标评估新知识掌握程度，涵盖视觉问答、图表理解等多个子领域。
• Model Agnosticism：实验表明该框架适用于不同架构和规模的 base models（如 LLaVA、Qwen-VL 等），显示出良好的通用性和迁移能力。

行业启示

• 对于希望快速迭代产品功能的 AI 团队而言，RoCo-ACE 提供了一种低开销、高保真的模型更新路径，尤其适用于医疗、法律等专业领域需频繁引入最新法规或案例的场景。
• 随着边缘设备对轻量化在线学习能力的需求增长，此类无需大规模重训即可实现局部优化的方法将成为下一代自适应系统的核心技术组件之一。
• 未来可进一步探索将 RoCo-ACE 扩展至语音、视频等多模态融合任务，并结合因果推理模块提升决策一致性，构建真正具备“记忆—适应—进化”闭环能力的通用人工智能系统。