Look Before You Leap: Pre-Action Verification for LLM Agents
Pre-action verification using cheap deterministic checks before LLM agent actions execute can effectively prevent silent failures that produce plausible but incorrect effects A static verifier for shell commands (9,930 commands, 482 tools) catches 95.8% of invalid commands at 10.0% false-positive rate, with syntax/binary checks being oracle-exact Code edit benchmarks reveal a critical format split: content-anchored formats (search/replace, diff) fail cleanly, while location-anchored formats (lin
Analysis
TL;DR
- Pre-action verification using cheap deterministic checks before LLM agent actions execute can effectively prevent silent failures that produce plausible but incorrect effects
- A static verifier for shell commands (9,930 commands, 482 tools) catches 95.8% of invalid commands at 10.0% false-positive rate, with syntax/binary checks being oracle-exact
- Code edit benchmarks reveal a critical format split: content-anchored formats (search/replace, diff) fail cleanly, while location-anchored formats (line numbers, function names) fail silently—line numbers corrupt 99.1% of files under a one-line shift
- A "refuse-when-unsure" policy converts silent failures into recoverable ones, achieving 0.958 recall at 7.0% false positives with selective grounding
- The authors release both benchmarks, verifiers, and guards as open resources
Why It Matters
This work addresses a critical reliability gap in LLM agents: silent failures that produce incorrect but plausible results without raising errors, which are far more dangerous than loud failures. For AI practitioners building agentic systems, it provides a practical, low-cost oversight mechanism that can be integrated before action execution. The findings also offer concrete guidance on action format selection, showing that content-anchored formats dramatically outperform location-anchored ones for reliability.
Technical Details
- Shell command verification: A static verifier analyzes commands against tool help-text and binary signatures. Syntax and binary checks are oracle-exact (zero false positives, catching ~50% of errors), while flag checks are bounded by help-text coverage and account for all false positives
- Code edit benchmark: 640 edits across 224 files isolating the apply step, comparing content-anchored formats (search/replace, diff) against location-anchored formats (line numbers, function names)
- Anchor-and-verify applier: Records only 1 silent misapplication in 8,320 trials (0.01%), demonstrating near-perfect reliability when verification is applied before execution
- Refuse-when-unsure policy: A tunable abstention mechanism that trades applicability for safety, with selective grounding reaching 0.958 recall at 7.0% false positives
- Open release: Both benchmarks, verifiers, and guards are publicly released for community use
Industry Insight
- Agent frameworks should prioritize content-anchored action formats (search/replace, diffs) over location-anchored ones (line numbers) to minimize silent failure risk in production systems
- Pre-action verification should become a standard component in agentic pipelines, offering high recall at modest false-positive costs—particularly valuable in high-stakes domains where silent corruption is unacceptable
- The "refuse-when-unsure" paradigm offers a practical safety valve: rather than attempting potentially incorrect actions, agents can abstain and escalate, enabling recoverable failure modes that preserve system integrity
Disclaimer: The above content is generated by AI and is for reference only.