The Readout Shortcut: Positional Number Copying Dominates Arithmetic CoT Readout in Small Language Models

Background

The study examines how CoT prompting impacts arithmetic performance in small language models, specifically focusing on three instruction-tuned models (Qwen, Llama, Gemma) with 1-3 billion parameters trained on GSM8K. The research aims to understand if the primary benefit of CoT comes from logical reasoning or a simpler positional shortcut.

Key Points

• Positional Shortcut: The model often copies the number immediately preceding the answer delimiter instead of using intermediate steps for reasoning.
• Accuracy Contribution: The gold-answer presence accounts for 54-92 percentage points (89-92% of each model's teacher-forcing ceiling) in accuracy, indicating a significant role of this shortcut.
• Model Behavior: Qwen and Llama tend to copy novel distractors 87-95% of the time, while Gemma exhibits selective gating behavior. A head-level ablation test suggests architecture-specific configurations influence the model's behavior.
• Task-Specific Differences: On non-arithmetic tasks like BBH, shuffling retains much less accuracy, and content-selective gating emerges at 7-8 billion parameters.

Significance

Positional Shortcut Precedence: The findings highlight that in arithmetic tasks, a positional shortcut dominates the model's behavior, overshadowing logical reasoning. This shortcut is not just a minor factor but a key contributor to performance, especially when correct intermediate steps are present.

• Model Mechanism: The models' reliance on copying numbers indicates that for arithmetic problems, simpler mechanisms can achieve near-optimal accuracy, challenging the necessity of complex logical reasoning in small models.
• Impact on CoT Evaluation: Step-level faithfulness evaluations may be misleading as they conflate positional answer transport with genuine computation. This raises questions about how effectively CoT-based oversight functions and its true impact on model behavior.

The study underscores that while CoT prompting is often believed to rely on logical reasoning, the models' performance can be heavily influenced by a simpler, positional mechanism. This has implications for understanding how small language models process arithmetic tasks and suggests potential avenues for optimizing or simplifying such models.