The Menu Is an Execution Prior: State-Path Tool Menus for Online Agents
Introduces the "tool menu" concept: a short, ordered subset of tools shown to an agent before execution, restricting calls to only menu items Proposes State-Path Tool Menu framework that learns pre-execution routes from observable request state to desired outcome Uses an encoder to model tool executability, input-output dependencies, and recurring execution orders, combined with a retriever and reranker Achieves online success rate of 0.898 on ToolBench, up from 0.737, outperforming retrieval, r
Analysis
TL;DR
- Introduces the "tool menu" concept: a short, ordered subset of tools shown to an agent before execution, restricting calls to only menu items
- Proposes State-Path Tool Menu framework that learns pre-execution routes from observable request state to desired outcome
- Uses an encoder to model tool executability, input-output dependencies, and recurring execution orders, combined with a retriever and reranker
- Achieves online success rate of 0.898 on ToolBench, up from 0.737, outperforming retrieval, reranking, generation, and routing baselines without modifying the agent
- Demonstrates that 32 tools in the State-Path menu cover more complete chains than 128 tools in the official list, with gains persisting across different model capacities
Why It Matters
This work addresses a critical bottleneck in practical AI agent deployment: the explosion of tool libraries containing thousands of interfaces. By reframing tool selection as learning execution priors rather than relevance ranking, it enables agents to construct complete, ordered tool chains without agent-side modifications. This has direct implications for making agentic systems more reliable in real-world scenarios with large tool ecosystems.
Technical Details
- Tool Menu: A constrained, ordered subset of tools presented to the agent pre-execution; the agent can only invoke tools within this menu
- State-Path Learning: The framework learns routes from observable request state to desired outcomes, with an encoder that captures (1) which tools are executable from the current state, (2) how tool outputs satisfy later inputs, and (3) recurring orderings from training paths
- Retriever + Reranker Architecture: The retriever identifies an executable entry point, missing-input producers, and the final action; the reranker then orders producers before consumers
- Benchmark Results: On ToolBench, the approach raises online success from 0.737 to 0.898, surpassing retrieval, reranking, generation, and routing baselines while remaining executor-agnostic
- Efficiency: The State-Path menu with 32 tools covers more complete execution chains than the official list with 128 tools, demonstrating superior tool selection density
Industry Insight
- Tool selection should shift from relevance-based ranking to execution-path-aware curation, as the most relevant tool is not always the most useful first step in a multi-step task
- The executor-agnostic nature of this approach means existing agent frameworks can adopt state-path menus as a drop-in improvement without architectural overhauls
- As tool libraries continue to grow into the thousands, the gap between available tools and usable tool subsets will widen, making execution-prior-based selection increasingly critical for agent reliability
Disclaimer: The above content is generated by AI and is for reference only.