Capacity-Dependent Effects of Data Selection for Reasoning
Likelihood-based data selection for reasoning SFT is not universally optimal; its effectiveness depends critically on model capacity and training duration Small models (1.5B-3B) benefit more from high-likelihood data with faster, more stable early improvements ("Fast-Fit" regime) Larger models (8B) gain more from low-likelihood data when training continues longer ("Slow-Gain" regime), as they can better absorb challenging supervision The paper introduces a "Fast-Fit / Slow-Gain" framework explai
Analysis
TL;DR
- Likelihood-based data selection for reasoning SFT is not universally optimal; its effectiveness depends critically on model capacity and training duration
- Small models (1.5B-3B) benefit more from high-likelihood data with faster, more stable early improvements ("Fast-Fit" regime)
- Larger models (8B) gain more from low-likelihood data when training continues longer ("Slow-Gain" regime), as they can better absorb challenging supervision
- The paper introduces a "Fast-Fit / Slow-Gain" framework explaining how data difficulty, data span, and student capacity jointly govern knowledge transfer
- Effective data selection strategies must account for both model scale and available compute budget rather than relying on a single universal heuristic
Why It Matters
This research challenges a widely held assumption in the AI community that high-likelihood responses are always preferable for supervised fine-tuning, which has significant implications for how practitioners design data selection pipelines. For AI engineers working on reasoning models, understanding this capacity-dependent dynamic can prevent wasted compute on suboptimal data strategies and help match data difficulty to model capability. The findings are particularly relevant as the industry scales up reasoning models and seeks efficient fine-tuning methodologies.
Technical Details
- Experimental setup: Controlled experiments on mathematical reasoning with student models ranging from 1.5B to 8B parameters, using supervision generated by stronger teacher models
- Key finding: A clear capacity-dependent "Fast-Fit / Slow-Gain" pattern emerges—high-likelihood data accelerates early training for smaller models, while low-likelihood data yields superior long-term gains for larger models
- Learning dynamics analysis: Small models trained on low-likelihood data tend to fail to absorb the supervision and collapse into shallow or repetitive behaviors, whereas larger models can effectively navigate toward the teacher distribution
- Theoretical contribution: A capacity-constrained theoretical framework of distillation is proposed, formalizing how data difficulty, data span, and student capacity interact to govern transfer effectiveness
- Domain: Mathematical reasoning tasks with supervised fine-tuning (SFT) using candidate responses of varying likelihood scores relative to the student model
Industry Insight
- Data selection pipelines should be dynamically configured based on model size and training budget rather than applying a one-size-fits-all likelihood threshold; practitioners should consider mixing high- and low-likelihood data proportionally to model capacity
- For resource-constrained teams training smaller models, prioritizing high-likelihood data will yield faster ROI, while teams with larger models and extended training windows should invest in harder, lower-likelihood samples for better final performance
- The "Fast-Fit / Slow-Gain" framework suggests that early stopping on small models with high-likelihood data is reasonable, but larger models require longer training schedules to realize the full benefit of challenging data—training duration should be treated as a co-parameter with data selection strategy
Disclaimer: The above content is generated by AI and is for reference only.