First Make It Playable, Then Make It Good: Staged Interaction Learning for Small Dialogue-Game Agents
Qwen-GuidePlay-2B is a 2B-parameter dialogue-game agent fine-tuned from Qwen3.5-2B using a staged three-step SFT pipeline The approach achieves 57.12 clemscore and 42.68 statscore on Playpen validation, with a +36 clemscore delta over the base model in official challenge results (second-highest among all submitted systems) The three-stage training consists of: successful-trajectory SFT, weighted turn-level SFT, and teacher-guided SFT where the teacher only fixes formatting and evaluates examples
Analysis
TL;DR
- Qwen-GuidePlay-2B is a 2B-parameter dialogue-game agent fine-tuned from Qwen3.5-2B using a staged three-step SFT pipeline
- The approach achieves 57.12 clemscore and 42.68 statscore on Playpen validation, with a +36 clemscore delta over the base model in official challenge results (second-highest among all submitted systems)
- The three-stage training consists of: successful-trajectory SFT, weighted turn-level SFT, and teacher-guided SFT where the teacher only fixes formatting and evaluates examples without generating new gold actions
- Procedurally heavy methods like replay-repair and hard-example mining provided no benefit, suggesting careful data curation is more effective than aggressive algorithmic changes for small models
- Both the model and code are released for reproducibility
Why It Matters
This work demonstrates that small language models (2B parameters) can achieve competitive performance in complex dialogue-game settings through a carefully designed staged fine-tuning strategy, challenging the assumption that only large models can handle interactive tasks effectively. The findings offer a practical blueprint for practitioners looking to build efficient dialogue agents without relying on massive compute or procedurally expensive training techniques.
Technical Details
- Base model: Qwen3.5-2B, fine-tuned via three sequential SFT stages on the Playpen dialogue-game dataset
- Stage 1: Supervised fine-tuning on only successful game trajectories to establish baseline playability
- Stage 2: Weighted turn-level SFT to improve per-turn decision-making quality
- Stage 3: Teacher-guided SFT using a larger model solely for formatting correction and example evaluation — the teacher does not generate new gold-standard actions, avoiding distribution shift
- Benchmarks: Playpen validation set with clemscore (57.12) and statscore (42.68); official challenge results showed second-highest clemscore delta (~+36 over base)
- Negative findings: Replay-repair and hard-example mining approaches did not improve performance, indicating that data curation quality outweighs procedural complexity for small models
Industry Insight
- Small models can be highly competitive in interactive dialogue tasks when trained with staged, curriculum-style fine-tuning — organizations should consider investing in data curation pipelines rather than scaling model size alone
- The teacher-only-formatting approach avoids the risk of policy drift that can occur when larger models generate training labels, offering a safer distillation paradigm for production systems
- The failure of replay-repair and hard-example mining suggests that for resource-constrained settings, simpler and more deliberate training strategies may outperform complex algorithmic interventions, redirecting engineering effort toward data quality over training sophistication
Disclaimer: The above content is generated by AI and is for reference only.