Think Big, Search Small: Where Capacity Matters in Hierarchical Search Agents?
Role factorization in hierarchical search agents consistently outperforms single-agent baselines, yielding significant improvements in exact match scores across various model scales. Capacity sensitivity is highly asymmetric; scaling the delegation backbone provides substantial performance gains (~11 EM points), whereas scaling the execution sub-agent offers minimal benefit (~2.6 EM points). A distilled 1.7B-parameter executor achieves parity with frontier sub-agents in accuracy while reducing t
Analysis
TL;DR
- Role factorization in hierarchical search agents consistently outperforms single-agent baselines, yielding significant improvements in exact match scores across various model scales.
- Capacity sensitivity is highly asymmetric; scaling the delegation backbone provides substantial performance gains (~11 EM points), whereas scaling the execution sub-agent offers minimal benefit (~2.6 EM points).
- A distilled 1.7B-parameter executor achieves parity with frontier sub-agents in accuracy while reducing token consumption by 37%, optimizing the efficiency-accuracy trade-off.
Why It Matters
This research challenges the common practice of using uniformly scaled models for all agent roles, providing empirical evidence that intelligent resource allocation is critical for performance. It offers a practical blueprint for developers to optimize cost and latency by focusing computational power on task decomposition rather than retrieval execution.
Technical Details
- Role Factorization: The study decomposes hierarchical search into three distinct roles: delegation (task decomposition), execution (retrieval and evidence extraction), and answer generation (held fixed as a control).
- Experimental Design: Controlled capacity sweeps were conducted on both delegation and execution axes across five multi-hop QA benchmarks and six different model scales.
- Key Findings: Delegation capacity was identified as the primary bottleneck, with scaling it driving the majority of performance improvements compared to execution scaling.
- Optimization Strategy: Quality-filtered trajectory distillation was used to train a smaller 1.7B-parameter executor, demonstrating that smaller models can match larger ones when optimized correctly.
Industry Insight
AI practitioners should prioritize investing in larger, more capable models for the planning and decomposition stages of multi-agent systems, while utilizing smaller, distilled models for execution tasks to maximize efficiency. This asymmetric scaling strategy can significantly reduce operational costs without compromising answer accuracy. Future agent architectures should explicitly separate and optimize these roles rather than relying on homogeneous model deployments.
Disclaimer: The above content is generated by AI and is for reference only.