Rethinking Self-Evolution: A Constrained Exploration-Exploitation Process for Mitigating Skill Overfitting 重新思考自我进化：一种受限的探索-利用过程以减轻技能过拟合

TL;DR

• SkillBoost is a three-stage framework for skill self-evolution in LLM agents that mitigates overfitting and regression.
• It uses structured exploitation to localize failures, prior-guided exploration to generate repair candidates, and verified acceptance to commit improvements within a regression bound.
• Experiments across 23 model-benchmark configurations show SkillBoost achieves state-of-the-art performance while outperforming human-crafted and LLM-generated skills.
• Optimized skills demonstrate transferability, enabling reuse by other agents on similar tasks.

Why It Matters

This work addresses a critical challenge in deploying LLM agents: the risk of overfitting during skill optimization from limited real-world trajectories. By balancing exploration and exploitation through constrained search, SkillBoost enables safer, more generalizable skill evolution—essential for robust real-world agent deployment where catastrophic forgetting or over-specialization can severely limit utility.

Technical Details

• Structured Exploitation: Identifies specific components within a skill that contribute to observed failures, allowing targeted edits rather than global retraining.
• Prior-Guided Exploration: Leverages the inherent knowledge within the LLM to propose diverse candidate repairs, reducing reliance on exhaustive random search.
• Verified Acceptance: Only accepts new skill variants if they improve task performance without degrading past success beyond a predefined regression threshold.
• Evaluation Scope: Tested across 23 combinations of models and benchmarks, demonstrating consistent improvement over baselines including human-designed and naively evolved skills.
• Transfer Learning: Skills optimized via SkillBoost were successfully transferred to other agents performing similar tasks, indicating modularity and generalizability.

Industry Insight

SkillBoost offers a practical pathway for enterprises deploying autonomous LLM agents to iteratively refine behaviors without sacrificing stability or requiring massive retraining cycles. Its emphasis on bounded exploration and verifiable updates makes it suitable for safety-critical domains like healthcare, finance, or robotics, where uncontrolled adaptation could lead to harmful outcomes. Additionally, its transfer capability suggests potential for collaborative agent ecosystems where shared skill libraries accelerate collective learning across organizations or platforms.

TL;DR

• 提出SkillBoost框架，通过约束探索与利用的平衡解决LLM技能自进化中的过拟合问题。
• 结构化 exploitation 定位失败点并编辑技能组件，prior-guided exploration 生成多样化修复候选，verified acceptance 确保性能提升且不回退。
• 在23种模型-基准配置上实现SOTA，优于人工和LLM生成的技能，且优化技能可跨agent复用。

为什么值得看

该工作针对LLM agent技能积累的核心挑战——数据驱动优化的过拟合风险，提供了一种系统化的自进化机制。对构建具备持续学习能力的智能体具有直接指导意义，尤其在减少人工干预和提升泛化能力方面。

技术解析

• 核心矛盾：过度利用现有轨迹导致当前批次过拟合，无约束探索则引发先前任务的性能退化（regression），需在探索与利用间建立动态约束。
• 三阶段流程：1）结构化exploitation将失败归因至可编辑的技能模块；2）prior-guided exploration利用LLM先验知识生成多方案修复；3）verified acceptance仅在验证性能提升且未突破回归阈值时提交更新。
• 实验设计：覆盖23组不同模型与基准组合，对比基线包括人工设计技能、原始LLM生成技能及无约束进化方法，同时测试技能迁移能力。
• 关键指标：不仅关注最终任务成功率，更强调技能演化过程中的稳定性（避免回退）与泛化性（跨agent/任务复用）。

行业启示

• LLM agent的长期能力演进需从“参数微调”转向“技能状态管理”，将技能视为可迭代优化的独立单元而非静态输出。
• 技能进化必须内置防退化机制（如回归边界约束），否则短期性能提升可能以牺牲历史能力为代价，影响系统可靠性。
• 跨agent技能共享潜力巨大，经约束进化优化的技能包可作为标准化组件部署于同类任务场景，降低重复开发成本。