Load-Bearing Context: The Question Damage Score for Evaluating Context Reliance in Linguistic Reasoning
Introduces a diagnostic framework using UK Linguistics Olympiad puzzles to measure whether LLMs derive answers from provided context or rely on prior knowledge Proposes the Question Damage Score, which classifies puzzles as fragile or robust based on the impact of removing individual context examples Two deletion strategies are used: uniform random deletion and targeted deletion inspired by error-correcting codes to remove structurally load-bearing context Three frontier LLMs rarely abstain when
Analysis
TL;DR
- Introduces a diagnostic framework using UK Linguistics Olympiad puzzles to measure whether LLMs derive answers from provided context or rely on prior knowledge
- Proposes the Question Damage Score, which classifies puzzles as fragile or robust based on the impact of removing individual context examples
- Two deletion strategies are used: uniform random deletion and targeted deletion inspired by error-correcting codes to remove structurally load-bearing context
- Three frontier LLMs rarely abstain when instructed to do so upon insufficient information, often producing correct answers even after critical context is removed
- The framework opens avenues for causal interventions, stopping-set analysis, contamination studies, and mechanistic interpretability research
Why It Matters
This work addresses a critical gap in evaluating LLMs: distinguishing genuine context-based reasoning from memorization or prior knowledge retrieval. As LLMs are increasingly deployed in domains requiring strict adherence to provided context (e.g., legal, medical, technical support), understanding their reliance patterns is essential for building trustworthy systems.
Technical Details
- Dataset: 53 UK Linguistics Olympiad puzzles, self-contained problems where all answers derive solely from expert-designed context examples without requiring external knowledge
- Question Damage Score: A formalized metric quantifying the impact of removing individual context examples on model performance, enabling classification of puzzles as fragile (highly dependent on specific context) or robust (resilient to context removal)
- Deletion strategies: (1) Uniform random deletion of single context examples, and (2) targeted deletion inspired by error-correcting codes, designed to remove structurally load-bearing examples that uniquely carry necessary information for specific questions
- Evaluation setup: Three frontier LLMs tested under explicit instructions to abstain when information is insufficient, measuring both abstention rates and answer correctness after context removal
- Extended analysis potential: The framework supports causal interventions, stopping-set analysis, targeted contamination studies, and mechanistic interpretability investigations
Industry Insight
- Current frontier LLMs exhibit concerning overconfidence: they rarely acknowledge information gaps and often generate plausible but potentially unsupported answers, posing risks in high-stakes applications where grounding in provided context is critical
- The Question Damage Score and diagnostic framework offer a practical evaluation tool for organizations to assess context reliance in their models before deployment in domain-specific applications
- The connection to error-correcting codes for targeted deletion suggests a novel methodological bridge between information theory and NLP evaluation, potentially inspiring new benchmark design approaches
Disclaimer: The above content is generated by AI and is for reference only.