Refusal Is Not Robustness: Auditing Confident Fabrication in Large Language Models on a Provably Uninformative Clinical Pain Speech Transcript
The study introduces a novel benchmark using the TAME Pain speech corpus to audit LLM fabrication, establishing that transcripts contain no lexical pain information, making any pain score inference from text alone unsupported by evidence Six of seven tested models correctly abstained under cooperative prompting but showed prompt-dependent abstention under authority-framed prompts, with abstention rates ranging from 0.18 to 1.00 across equivalent phrasings Gemini 2.5 Flash and Llama 3.1 8B exhibi
Analysis
TL;DR
- The study introduces a novel benchmark using the TAME Pain speech corpus to audit LLM fabrication, establishing that transcripts contain no lexical pain information, making any pain score inference from text alone unsupported by evidence
- Six of seven tested models correctly abstained under cooperative prompting but showed prompt-dependent abstention under authority-framed prompts, with abstention rates ranging from 0.18 to 1.00 across equivalent phrasings
- Gemini 2.5 Flash and Llama 3.1 8B exhibited alarming confident fabrication rates of 0.53 and 0.76 respectively, far exceeding the 0.15 maximum of other models
- Models achieved near-perfect accuracy (0.939–1.00) on positive control tasks where pain ratings were explicitly spoken, confirming capability when information is actually present
- No significant demographic effects were observed in forced responses (all p ≥ 0.20), suggesting fabrication is model-dependent rather than context-dependent
Why It Matters
This research addresses a critical gap in AI safety evaluation: existing hallucination benchmarks cannot distinguish between appropriate abstention and unsupported predictions because they rarely prove the model could not have known the answer. For AI practitioners building clinical or high-stakes applications, the findings reveal that refusal behavior is fragile and highly sensitive to prompt framing, meaning systems that appear safe under one prompting style may produce confidently fabricated outputs under another.
Technical Details
- Dataset: TAME Pain speech corpus with 5,750 "no signal" Harvard Sentence utterances (no lexical pain information in transcripts) and 1,294 "signal" pain statement utterances with explicitly spoken pain ratings
- Validation approach: Pain was recoverable from acoustic features (AUC 0.622) but transcript-based prediction was near chance (AUC 0.489), proving ASR-transcribed text contains no usable pain signals
- Models evaluated: Seven large language models tested under both cooperative and authority-framed prompting conditions
- Metrics: Abstention rates, extraction accuracy, expected calibration error (≤0.100), and confident fabrication rates
- Key finding: Under cooperative prompting, six models abstained on nearly all no-signal transcripts; under authority framing, the same models showed wildly variable abstention (0.18–1.00) depending on prompt phrasing
Industry Insight
- Prompt engineering is insufficient for safety: Since abstention rates varied dramatically across equivalent prompt phrasings, relying on prompt design alone cannot guarantee models will refuse unsupported queries—robust architectural safeguards are needed
- Confident fabrication is a model-specific risk: Gemini 2.5 Flash and Llama 3.1 8B's high fabrication rates suggest certain model architectures or training approaches may be more prone to generating confident but unsupported outputs, warranting careful model selection for clinical applications
- Benchmark design standards should evolve: The field needs benchmarks that provably exclude answerability from the input modality being evaluated, rather than relying on static datasets where information might implicitly exist
Disclaimer: The above content is generated by AI and is for reference only.