Learning Evidence Sufficiency Boundaries for Selective Answering in Grounded Multi-Hop QA
Introduces Evidence Sufficiency Boundary Training, a generation-native framework that teaches models to abstain when evidence is insufficient and answer only when context reaches a sufficiency threshold Addresses the core challenge in multi-hop QA where partial evidence can make unsupported answers appear plausible Achieves flip accuracy of 0.807 on boundary localization, outperforming a token-level abstention baseline (0.781), with the lowest unsupported-answer rate of 0.095 on external non-ans
Analysis
TL;DR
- Introduces Evidence Sufficiency Boundary Training, a generation-native framework that teaches models to abstain when evidence is insufficient and answer only when context reaches a sufficiency threshold
- Addresses the core challenge in multi-hop QA where partial evidence can make unsupported answers appear plausible
- Achieves flip accuracy of 0.807 on boundary localization, outperforming a token-level abstention baseline (0.781), with the lowest unsupported-answer rate of 0.095 on external non-answerable sets
- Combines four training components: level supervision, boundary flip margin, post-boundary stability, and answer recall protection
- Evaluated on HotpotQA, 2WikiMultiHopQA, and MuSiQue using Qwen2.5-3B-Instruct with LoRA adaptation, maintaining competitive raw QA F1 while improving selective answering
Why It Matters
This work directly addresses a critical reliability gap in grounded QA systems: the tendency to answer confidently even when evidence is incomplete or misleading. For AI practitioners building production question-answering systems, the ability to selectively abstain rather than hallucinate is essential for trust and safety. The paper provides a practical training framework that can be adapted to improve the dependability of retrieval-augmented and multi-hop QA pipelines.
Technical Details
- Evidence Sufficiency Boundary Training constructs ordered evidence chains from multi-hop QA datasets and directly supervises the abstain-to-answer transition, rather than relying on token-level confidence thresholds
- The framework combines four objectives: level supervision (training at each evidence depth), boundary flip margin (encouraging a clear transition zone), post-boundary stability (preventing answer changes when redundant evidence is added), and answer recall protection (ensuring sufficient answers are still produced)
- Evidence chains are built from three benchmark datasets: HotpotQA, 2WikiMultiHopQA, and MuSiQue
- Evaluation uses chain-level metrics, raw QA F1, and unsupported-answer rates on external non-answerable sets
- Implementation uses Qwen2.5-3B-Instruct as the base model with LoRA adaptation, keeping the approach parameter-efficient
Industry Insight
- Selective answering should be treated as a first-class training objective rather than a post-hoc filtering step; models trained to recognize evidence boundaries will reduce hallucination rates in production QA systems
- The four-component training framework is modular and can be adapted to other grounded reasoning tasks beyond multi-hop QA, including fact-checking and medical/legal question answering where evidence sufficiency is critical
- The modest but consistent improvements (flip accuracy +2.6%, unsupported-answer rate reduction) suggest that boundary-aware training is a reliable optimization path, especially when combined with retrieval-augmented generation pipelines
Disclaimer: The above content is generated by AI and is for reference only.