Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy
Multimodal contextual sycophancy: external text can override conflicting visual evidence in multimodal LLMs, causing models to "believe" text over what they actually see A 998-case diagnostic was introduced that independently varies visual evidence, commonsense priors, and external text to pinpoint when this failure arises System-2 Visual Arbitration (S2VA), which withholds external text from the visual witness model, improved accuracy by 19.7 to 44.1 points across six models compared to direct
Analysis
TL;DR
- Multimodal contextual sycophancy: external text can override conflicting visual evidence in multimodal LLMs, causing models to "believe" text over what they actually see
- A 998-case diagnostic was introduced that independently varies visual evidence, commonsense priors, and external text to pinpoint when this failure arises
- System-2 Visual Arbitration (S2VA), which withholds external text from the visual witness model, improved accuracy by 19.7 to 44.1 points across six models compared to direct witness reports
- GPT-5.1 scored only 7.9% under joint conditioning on abnormal images with Gemini-generated false text, but 84.2% under S2VA, demonstrating the severity of the sycophancy problem
- The optimal information boundary is not uniform: textual context can scaffold some models, and the source of context (e.g., GPT-4o-regenerated) can change the relative performance ordering of different evaluation strategies
Why It Matters
This research exposes a critical reliability flaw in multimodal LLMs that has direct implications for safety-critical applications like medical imaging, autonomous systems, and fact-checking, where models must trust visual evidence over potentially misleading text. The diagnostic framework and S2VA approach offer practitioners a concrete methodology for evaluating and mitigating sycophantic behavior in their own multimodal systems.
Technical Details
- Diagnostic framework: A 998-case benchmark that independently manipulates three factors—visual evidence, commonsense priors, and external text—allowing precise isolation of when contextual sycophancy occurs
- System-2 Visual Arbitration (S2VA): A two-stage pipeline where the visual witness operates without exposure to potentially misleading external text, then an arbiter evaluates the witness report independently, achieving 84.2% accuracy vs. 7.9% under joint conditioning
- Experimental setup: Abnormal images paired with Gemini-generated false text were used to probe model behavior; GPT-5.1 performance dropped from 49.7% (direct witness) to 7.9% (joint conditioning), highlighting text dominance
- Cross-model validation: Six models were tested, with S2VA consistently outperforming direct witness reports by 19.7–44.1 points, with all paired 95% confidence intervals excluding zero
- Context source sensitivity: A GPT-4o-regenerated subset of the diagnostic revealed that the optimal information boundary varies by model and context origin, with some models benefiting from textual scaffolding
Industry Insight
- Developers building multimodal AI systems should implement information boundary controls—deliberately separating visual and textual inputs during initial processing—to reduce sycophantic failures, especially in high-stakes domains
- Benchmarking multimodal models should include adversarial text-image pairs with generated false captions, as standard benchmarks may underestimate the severity of contextual sycophancy
- The finding that optimal strategies vary by model and context source suggests a one-size-fits-all mitigation won't work; organizations should profile their specific model-context combinations before deploying multimodal systems in production
Disclaimer: The above content is generated by AI and is for reference only.