trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories
Outcome-only LLM judges are structurally blind to agents that reach correct answers through flawed reasoning paths, catching only 45% of silent faults A step-rubric judge achieves 77% silent fault recall with zero false alarms but at 3x the computational cost of outcome-only evaluation No evaluated judge (including step-rubric and self-consistency ensembles) detected an invented promise appended to an otherwise perfect trajectory, with the step judge missing it 82% of the time Self-consistency e
Analysis
TL;DR
- Outcome-only LLM judges are structurally blind to agents that reach correct answers through flawed reasoning paths, catching only 45% of silent faults
- A step-rubric judge achieves 77% silent fault recall with zero false alarms but at 3x the computational cost of outcome-only evaluation
- No evaluated judge (including step-rubric and self-consistency ensembles) detected an invented promise appended to an otherwise perfect trajectory, with the step judge missing it 82% of the time
- Self-consistency ensembles tripled evaluation cost while providing no meaningful improvement in detection accuracy
- The authors advocate for stratifying judge evaluation recall by outcome survival and release their deterministic test environment, fault injector, and full analysis pipeline
Why It Matters
This research exposes a critical blind spot in how the AI industry evaluates LLM agents: the near-universal reliance on outcome-only judging masks dangerous failure modes where agents produce correct answers through incorrect or unsafe reasoning. For practitioners building production agent systems, these findings suggest that current evaluation pipelines may be significantly overestimating agent reliability, particularly for high-stakes applications where the path to an answer matters as much as the answer itself.
Technical Details
- Test environment: A deterministic tool-using support-desk environment with a scripted oracle policy that always solves tasks, enabling precise ground-truth construction for evaluation
- Fault injection methodology: A fault injector breaks exactly one component at a known step, stratifying faults into "loud" (customer-visible outcome affected) and "silent" (outcome survives despite internal failure) categories
- Five judges evaluated: Programmatic rules, outcome-only LLM judge, step-rubric judge at two model sizes, and a self-consistency ensemble, scored across detection, step localization, fault typing, calibration, and cost metrics over 400 trajectories
- Key quantitative findings: Outcome-only judge catches 84% of loud faults but only 45% of silent ones while generating 33% false positives on correct trajectories; step-rubric reaches 77% silent recall with zero false alarms; invented promises evade all judges (rules: 100% miss rate, step judge: 82% miss rate)
- Open release: The authors release the environment, fault injector, all raw verdicts, and an offline analysis pipeline that reconstructs every reported metric
Industry Insight
- Organizations should adopt step-level evaluation for safety-critical agent deployments, accepting the 3x cost premium as insurance against silent failures that outcome-only metrics systematically miss
- The failure of all judges to detect appended invented promises suggests that current evaluation benchmarks may be gamed or insufficiently stress-tested; practitioners should develop adversarial evaluation suites that include hallucinated output injection
- The stratification framework (separating loud vs. silent fault detection) should become a standard reporting metric in agent evaluation papers, as aggregate numbers obscure the critical distinction between catching obvious failures versus subtle reasoning corruption
Disclaimer: The above content is generated by AI and is for reference only.