Can a Model Catch Its Own Hallucinations for Free?: Label-Free Doubt Signals Hold Their Own Against a Labelled Dataset for Abstention
LLMs can use their own internal confidence signals as a near-free substitute for labelled datasets when learning when to abstain from answering Fine-tuning with LoRA to answer on high confidence and say "I'm not sure" on low confidence matches the performance of label-supervised abstention tuning across six open-weights models (1B–8B) The performance gain comes from improved calibration, not rote memorization, as shown by a control experiment that drilled hard examples without abstention The lab
Analysis
TL;DR
- LLMs can use their own internal confidence signals as a near-free substitute for labelled datasets when learning when to abstain from answering
- Fine-tuning with LoRA to answer on high confidence and say "I'm not sure" on low confidence matches the performance of label-supervised abstention tuning across six open-weights models (1B–8B)
- The performance gain comes from improved calibration, not rote memorization, as shown by a control experiment that drilled hard examples without abstention
- The label-free doubt signal has a critical blind spot: it cannot flag confidently wrong facts (hallucinations where the model is confidently incorrect)
- Correctness was adjudicated by an independent judge model, ensuring unbiased evaluation across short-form factual question-answering tasks
Why It Matters
This research directly addresses one of the most pressing challenges in deploying LLMs: hallucination and overconfident incorrect outputs. By demonstrating that models can self-calibrate without expensive labelled datasets, it opens a practical path for reducing harmful AI outputs at scale, especially for organizations that lack the resources to curate large annotated datasets for abstention tuning.
Technical Details
- Method: LoRA-based fine-tuning applied to six open-weights models (1B–8B, two families) to condition responses on frozen confidence scores—answer when confidence is high, output "I'm not sure" when low.
- Evaluation: Short-form factual question answering, with correctness adjudicated by an independent judge model rather than ground-truth labels alone.
- Comparison: Label-free abstention was benchmarked against label-supervised abstention tuning, finding no statistically detectable difference at matched coverage.
- Control experiment: A variant that trained on hard examples without abstention showed no improvement, confirming the gain is attributable to calibration rather than memorization.
- Limitation identified: Confidently wrong facts (high-confidence hallucinations) remain undetectable by this approach, representing a fundamental blind spot.
Industry Insight
- Organizations can implement abstention mechanisms without the cost and effort of building labelled correctness datasets, significantly lowering the barrier to safer LLM deployment.
- The finding that calibration—not memorization—drives the improvement suggests that confidence-aware fine-tuning should be prioritized over brute-force data collection strategies for reliability.
- Practitioners should be aware that label-free doubt signals cannot catch confident hallucinations; hybrid approaches combining confidence thresholds with external verification remain necessary for high-stakes applications.
Disclaimer: The above content is generated by AI and is for reference only.