When Auditors Fabricate: Batch-Size Degradation and Confident Hallucination in LLM Detection of Planted Document Contamination
LLMs used as automated document auditors show severe batch-size degradation, with detection dropping from 60% on small batches to just 2.8% on large batches The failure mode is fabrication, not abstention: models confidently invent contaminants that never existed in source documents Detection performance varies by contamination type, with absurd insertions recovered at 75% while plausible corruptions (semantic reversals, typographical errors) are missed at 50% The study evaluated Google Gemini 3
Analysis
TL;DR
- LLMs used as automated document auditors show severe batch-size degradation, with detection dropping from 60% on small batches to just 2.8% on large batches
- The failure mode is fabrication, not abstention: models confidently invent contaminants that never existed in source documents
- Detection performance varies by contamination type, with absurd insertions recovered at 75% while plausible corruptions (semantic reversals, typographical errors) are missed at 50%
- The study evaluated Google Gemini 3.0 Pro on a contaminated corpus of 150 academic papers with 450 injected contaminants across three types
- Safe deployment requires bounded batch sizes, direct content injection, and mechanical verification of every reported finding against source text
Why It Matters
This research reveals a critical reliability gap in using LLMs for automated auditing and quality control tasks, demonstrating that models don't gracefully degrade under scale but instead produce deceptive, confident hallucinations. For AI practitioners building detection or verification systems, these findings underscore that batch processing at scale introduces systemic failure modes that are far more dangerous than simple omission.
Technical Details
- Dataset: A contaminated corpus of 150 academic papers from supply chain management and medical research, with 450 known contaminants injected across three types: typographical corruption, semantic reversal, and absurd out-of-context insertion
- Model evaluated: Google Gemini 3.0 Pro tested on a 180-contaminant answer-key subset across 60 documents
- Prompting regimes: Three scales were tested—single document (50% recovery), small batch (60% recovery), and large batch (2.8% recovery)—showing a dramatic collapse at scale
- Failure mode: Rather than abstaining or reporting incomplete processing, the model fabricated contaminants including absurdities like "telepathic squirrel" and "quantum-powered toaster" that mimic the style of planted material but are entirely invented
- Contamination-type bias: Absurd insertions were recovered at 75% in completed evaluations, while semantic reversals and typographical corruptions were each recovered at only 50%, indicating that plausible corruptions most likely to occur in real-world settings are the hardest to detect
Industry Insight
- Organizations deploying LLMs for document auditing, compliance checking, or quality assurance must implement hard batch-size limits and never trust unverified model outputs at scale without mechanical cross-referencing against source text
- The deceptive nature of LLM failure—confident fabrication rather than honest omission—creates significant liability risks, as false positives and false negatives are indistinguishable without ground-truth verification
- Future system designs should treat LLM auditors as assistive tools requiring human-in-the-loop validation rather than autonomous decision-makers, particularly for high-stakes domains like medical research and regulatory compliance
Disclaimer: The above content is generated by AI and is for reference only.