Shared circuits predict whether LLMs generalize across formats in arithmetic reasoning
LLMs exhibit brittleness when generalizing from numeric arithmetic (e.g., "2+5") to verbal formats (e.g., "two plus five"), despite near-perfect numeric performance Attribution patching is used to independently localize the circuits recruited by models for numeric versus verbal arithmetic reasoning across English, Spanish, and Italian Circuit overlap between a model's numeric and verbal circuits predicts its generalization performance across verbal formats without requiring any labeled data The
Analysis
TL;DR
- LLMs exhibit brittleness when generalizing from numeric arithmetic (e.g., "2+5") to verbal formats (e.g., "two plus five"), despite near-perfect numeric performance
- Attribution patching is used to independently localize the circuits recruited by models for numeric versus verbal arithmetic reasoning across English, Spanish, and Italian
- Circuit overlap between a model's numeric and verbal circuits predicts its generalization performance across verbal formats without requiring any labeled data
- The circuit overlap metric accounts for relative difficulty across verbal formats, identifies which formats models generalize to best, and predicts item-level correctness, rivaling supervised probes
Why It Matters
This work provides a mechanistic, interpretable lens into a persistent weakness of LLMs: poor cross-format generalization in reasoning tasks. By showing that internal circuit overlap—not just surface-level features—predicts generalization ability, it offers practitioners a data-efficient diagnostic tool for evaluating model robustness without expensive labeled benchmarks.
Technical Details
- Method: Attribution patching is employed to localize the computational circuits each model recruits for solving numeric arithmetic problems (e.g., "2+5") versus verbal renditions (e.g., "two plus five," "dos más cinco," "due più cinque")
- Multilingual scope: The study evaluates generalization across three languages—English, Spanish, and Italian—allowing cross-linguistic comparison of circuit overlap and its predictive power
- Three-level prediction: Circuit overlap predicts (1) the relative difficulty ranking of the three verbal formats, (2) which verbal formats each model generalizes to best, and (3) which individual items are solved correctly
- Unsupervised advantage: The approach requires no labeled data and achieves predictive performance rivaling supervised probes, making it a cost-effective diagnostic for model internals
Industry Insight
- Model evaluation pipelines should incorporate mechanistic interpretability tools like attribution patching to diagnose generalization brittleness before deployment, rather than relying solely on benchmark accuracy
- The finding that circuit overlap predicts cross-format performance suggests that training strategies aimed at strengthening shared representational circuits could improve robustness to prompt variations—a key concern for real-world LLM applications
- The unsupervised nature of this diagnostic makes it scalable for continuous model monitoring across languages and domains without the overhead of creating labeled evaluation sets
Disclaimer: The above content is generated by AI and is for reference only.