Agent Skills: The Composition Cliff
Adding a second or third skill to an LLM agent reliably improves task performance, but adding a fourth skill causes a sharp decline in performance (the "composition cliff"). The SKILL.md format used across multiple platforms lacks dependency declarations, conflict signaling, and composition ordering mechanisms, leading to skills working against each other. Skills can improve outcomes on 68 out of 84 tasks but worsen them on 16, with roughly one in five skill activations producing results below t
Analysis
TL;DR
- Adding a second or third skill to an LLM agent reliably improves task performance, but adding a fourth skill causes a sharp decline in performance (the "composition cliff").
- The SKILL.md format used across multiple platforms lacks dependency declarations, conflict signaling, and composition ordering mechanisms, leading to skills working against each other.
- Skills can improve outcomes on 68 out of 84 tasks but worsen them on 16, with roughly one in five skill activations producing results below the baseline.
- Smaller models with well-chosen skills can outperform larger models without skills, as skills fill behavioral gaps that are explicit in smaller models and latent in larger ones.
- Skills written by models themselves perform poorly, often being too abstract or too specific to generalize effectively.
Why It Matters
This research highlights critical challenges in scaling LLM agents through skill composition, directly impacting how developers design and optimize multi-skill systems. Understanding the diminishing returns and potential conflicts when stacking skills is essential for building reliable, high-performance agents in real-world applications.
Technical Details
- SKILL.md Format: A YAML header (with fields like name, description, license, compatibility, metadata, and allowed-tools) followed by a Markdown body containing natural language instructions injected into the model’s context (typically the system prompt).
- SkillsBench Benchmark: Evaluates 84 tasks from real-world deployments, testing both frontier and smaller models while distinguishing between human-authored and model-generated skills.
- Performance Trends: Two to three skills yield peak gains (e.g., Claude Haiku with skills achieves ~28% success vs. Claude Opus without skills at 22%), but four or more skills reduce cumulative gains by 68%.
- Failure Modes: Model-authored skills tend to be either too abstract (lacking actionable guidance) or too specific (failing to generalize), performing below the no-skill baseline.
- Attention Dilution: Skills positioned in the middle of the context window receive less attention due to the "Lost in the Middle" phenomenon, worsening as total token count increases.
- Token Thresholds: Reasoning reliability declines sharply beyond ~3,000 tokens of instruction content; four skills averaging 400 tokens each approach this threshold before task-specific content is added.
- Non-Adherence Multiplication: If each skill has a 90% adherence rate in isolation, all four are simultaneously followed only 66% of the time (0.9⁴), compounding rapidly with more skills.
- Conflict Accumulation: The ScaledIF benchmark shows that instruction conflicts escalate nonlinearly—nearly unavoidable with 10+ instructions—leading to lower task success rates.
Industry Insight
Developers should prioritize careful curation and sequencing of skills rather than indiscriminate addition, focusing on complementary capabilities that avoid overlap or contradiction. Future tooling must incorporate formal mechanisms for declaring dependencies, detecting conflicts, and optimizing skill order within the SKILL.md ecosystem to mitigate the composition cliff. Additionally, investing in human-authored over model-generated skills will likely yield more consistent improvements in agent performance across diverse domains.
Disclaimer: The above content is generated by AI and is for reference only.