ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
ClaimReceipt introduces a claim-relative receipt specification and selective verifier that addresses two critical gaps in agent evaluation: evidence sufficiency (whether claims are recomputable from retained evidence) and coverage (whether retained records encompass the full committed experiment set) The system binds typed transaction evidence to a signed experiment manifest, returning one of three verdicts per claim: PASS, INVALID, or INCONCLUSIVE On 1,392 historical buyer-seller records, the C
Analysis
TL;DR
- ClaimReceipt introduces a claim-relative receipt specification and selective verifier that addresses two critical gaps in agent evaluation: evidence sufficiency (whether claims are recomputable from retained evidence) and coverage (whether retained records encompass the full committed experiment set)
- The system binds typed transaction evidence to a signed experiment manifest, returning one of three verdicts per claim: PASS, INVALID, or INCONCLUSIVE
- On 1,392 historical buyer-seller records, the CR-2 verifier reproduced all five manually labeled audit verdicts, exactly replayed 600 deterministic and 792 post-generation records, and achieved 11/11 semantic fault detection with zero false positives
- A prospective CR-3 epoch demonstrated that withholding a single terminal receipt yields INCONCLUSIVE_COVERAGE, while withholding private evidence openings preserves coverage but renders economic claims inconclusive—exactly matching preregistered predictions
- Receipt instrumentation imposes minimal overhead: 0.021% of model-inference time and 9.9 KB per transaction, though the frozen specification was found not yet unambiguous to independent readers
Why It Matters
Agent evaluation has lacked rigorous evidentiary standards, with generic logs and hash-linked transcripts failing to reliably verify whether claims are genuinely supported by retained evidence. ClaimReceipt provides a formal framework that enables auditors, researchers, and practitioners to cryptographically verify both the sufficiency and completeness of evaluation evidence, addressing a critical trust gap in multi-agent system benchmarking.
Technical Details
- Claim-relative receipt specification: Binds typed transaction evidence to a signed experiment manifest, enabling per-claim verification with three possible outputs: PASS, INVALID, or INCONCLUSIVE
- Specification freezing: The specification was cryptographically hashed (SHA-256) before implementation to prevent post-hoc modifications, establishing a reproducible baseline
- Empirical validation on 1,392 records: CR-2 verifier achieved perfect reproduction of manually labeled audit verdicts, exact replay of all deterministic and post-generation records, non-redundancy of all 13 declared field groups under ablations, and 100% semantic fault detection with no false positives
- Prospective CR-3 epoch: 30 assignments were committed before inference, terminal receipts were signed and chained, and private evidence was encrypted for auditor access—demonstrating real-world protocol behavior under selective evidence withholding
- Overhead metrics: Receipt instrumentation adds only 0.021% of model-inference time and 9.9 KB per transaction, making it practically deployable
Industry Insight
- The minimal overhead (0.021% inference time, 9.9 KB per transaction) suggests ClaimReceipt-style verification can be integrated into existing agent evaluation pipelines without significant performance penalties, making it viable for production benchmarking systems
- The finding that the frozen specification is not yet unambiguous to independent readers highlights an urgent need for improved specification legibility and standardization in cryptographic evaluation frameworks—researchers should prioritize human-readable, independently verifiable protocols
- The distinction between coverage verification and economic claim verification (where withholding private openings preserves protocol verification but renders economic claims inconclusive) suggests that agent evaluation frameworks should adopt layered verification strategies, with separate guarantees for structural integrity versus substantive claim validity
Disclaimer: The above content is generated by AI and is for reference only.