LLMs Know the Constraint But Do Not Use It: Activation Bottlenecks in Pragmatic Constraint Reasoning
LLMs internally encode pragmatic constraints symmetrically across prompts but fail to consistently route them into decisions, revealing a "conditional constraint activation" gap between knowledge and usage Probes on open-weight models decode constraints above 88%, proving knowledge is present; activation patching repairs one failure mode (+6.4 nats) but not the other (-0.07 nats) The quartet diagnostic (Knowledge, Symmetry, Routing, Repair) over 14 models identifies two distinct failure modes in
Analysis
TL;DR
- LLMs internally encode pragmatic constraints symmetrically across prompts but fail to consistently route them into decisions, revealing a "conditional constraint activation" gap between knowledge and usage
- Probes on open-weight models decode constraints above 88%, proving knowledge is present; activation patching repairs one failure mode (+6.4 nats) but not the other (-0.07 nats)
- The quartet diagnostic (Knowledge, Symmetry, Routing, Repair) over 14 models identifies two distinct failure modes in how LLMs handle implicit feasibility constraints
- No prompted intervention reaches the repair corner on the mitigation frontier—all interventions inflate conservative bias through a single mediation pathway (prerequisite mention)
- Hidden-constraint failure is fundamentally a routing problem, not a knowledge problem
Why It Matters
This research reframes a persistent failure mode in LLMs—ignoring implicit constraints—by demonstrating that the issue lies in activation routing rather than knowledge representation, which has direct implications for prompt engineering, model interpretability, and safety-critical deployments. For practitioners, it shows that standard prompting interventions cannot resolve this failure and may actually worsen conservative bias, necessitating architectural or mechanistic interventions instead.
Technical Details
- Conditional Constraint Activation Framework: The authors formalize four diagnostic dimensions—Knowledge (internal encoding of constraints), Symmetry (encoding consistency across constraint-present and absent prompts), Routing (whether encoded constraints are used in decisions), and Repair (whether donor activation patching can restore correct behavior).
- Quartet Diagnostic Across 14 Models: Empirical evaluation using probing and activation patching reveals two distinct failure modes; constraint decoding exceeds 88% accuracy on open-weight models, yet patching success is asymmetric (+6.4 nats for one mode, -0.07 nats for the other).
- Mitigation Frontier Analysis: Systematic evaluation of prompted interventions shows none achieve the repair corner; all interventions inflate conservative bias through a single mediation pathway (prerequisite mention), indicating a structural limitation of surface-level prompting.
- Activation Patching as Diagnostic Tool: Donor activation patching is used to causally test whether constraint representations can be routed into decisions, providing mechanistic evidence that knowledge exists but routing fails.
Industry Insight
- Prompt engineering alone cannot resolve hidden-constraint reasoning failures; organizations relying on LLMs for constraint-heavy tasks (e.g., compliance, planning, reasoning benchmarks) should invest in mechanistic interpretability and activation-level interventions rather than surface-level prompting.
- The asymmetric repairability across failure modes suggests that not all constraint reasoning failures are equivalent—diagnosing which mode is at play should guide whether to pursue architectural changes, fine-tuning, or alternative decoding strategies.
- The finding that all prompted interventions inflate conservative bias through prerequisite mention is a cautionary signal for safety-aligned systems: mitigation strategies may inadvertently reinforce undesirable default behaviors, warranting more nuanced intervention design.
Disclaimer: The above content is generated by AI and is for reference only.