Belief-Calibrated Optimization: An Explicit World Model for Agentic Optimization
Belief-Calibrated Optimization (BCO) introduces an explicit, persistent in-context world model that captures the coding agent's implicit beliefs about how environment edits will affect outcomes BCO continuously revises this world model document as new candidates are evaluated, enabling later optimization rounds to leverage accumulated environmental understanding BCO outperforms a matched control lacking the world model across five benchmarks spanning memory QA, tool-use QA, code-as-action app ag
Analysis
TL;DR
- Belief-Calibrated Optimization (BCO) introduces an explicit, persistent in-context world model that captures the coding agent's implicit beliefs about how environment edits will affect outcomes
- BCO continuously revises this world model document as new candidates are evaluated, enabling later optimization rounds to leverage accumulated environmental understanding
- BCO outperforms a matched control lacking the world model across five benchmarks spanning memory QA, tool-use QA, code-as-action app agents, and terminal agents, with gains persisting on held-out splits
- The BCO-optimized scaffold demonstrates transferability: after swapping the frozen target model, the selected scaffold still leads on tested tasks (except where context-window overruns occur)
- An offline ablation confirms the world model's content carries reusable predictive information, as a fresh predictor given the accumulated document forecasts environment responses more accurately than predictors with no document or falsified-content copies
Why It Matters
This work addresses a fundamental limitation in LLM agent optimization: the loss of learned environmental beliefs between optimization rounds. By making the agent's implicit understanding explicit and persistent, BCO enables more sample-efficient optimization and produces scaffolds that generalize across model swaps. For AI practitioners building agentic systems, this represents a practical path to improving agent performance without modifying the frozen base model.
Technical Details
- BCO operates within a standard iterative optimization loop where a coding agent reads current scores and execution traces, then edits source code to produce improved candidates
- The key innovation is a persistent in-context document that serves as an explicit world model, recording the agent's beliefs about what went wrong and which changes should help, with continuous revision as new evaluations arrive
- Evaluation spans five benchmarks: memory QA, tool-use QA, code-as-action app agents, and terminal agents, with performance measured via train passrate
- The method demonstrates cross-model generalization: after replacing the frozen target model while keeping the scaffold intact, BCO-selected scaffolds outperform controls on tested tasks
- An offline ablation uses a fresh predictor given the accumulated world model document to forecast environment responses, comparing against no-document and falsified-content baselines to isolate the contribution of document content versus structure
Industry Insight
- Explicit world modeling in agent optimization loops could become a standard technique for improving sample efficiency, as it prevents the repeated re-derivation of environmental knowledge across optimization rounds
- The transferability of BCO-optimized scaffolds across model swaps suggests that scaffold engineering can be decoupled from model selection, enabling more modular and cost-effective agent development pipelines
- Practitioners should monitor context-window constraints as a limiting factor; the authors note performance degradation when overruns leave the world model unfinished, indicating a trade-off between model depth and context management that will require engineering solutions as agents grow more complex
Disclaimer: The above content is generated by AI and is for reference only.