Speculative Correction: Draft-then-Refine Decoding for Diffusion Language Models
Introduces a plug-and-play "draft-then-refine" decoding pattern for diffusion language models (DLMs) that first generates a complete draft, then refines it bidirectionally Flash-Flash configuration (same model drafting and refining) improves GSM8K-384 accuracy from 0.848 to 0.899 and MBPP-384 from 0.545 to 0.693, while running 1.20x faster than the block-autoregressive baseline Mini-Flash speculative correction configuration achieves MATH-384 performance of 0.294 versus 0.300 for Flash while run
Analysis
TL;DR
- Introduces a plug-and-play "draft-then-refine" decoding pattern for diffusion language models (DLMs) that first generates a complete draft, then refines it bidirectionally
- Flash-Flash configuration (same model drafting and refining) improves GSM8K-384 accuracy from 0.848 to 0.899 and MBPP-384 from 0.545 to 0.693, while running 1.20x faster than the block-autoregressive baseline
- Mini-Flash speculative correction configuration achieves MATH-384 performance of 0.294 versus 0.300 for Flash while running 2.17x faster, demonstrating a training-free speedup route
- Causal ablations confirm that completed drafts provide useful initializations, with full global refinement benefiting GSM8K and local refinement capturing gains on MBPP and MATH
Why It Matters
This work demonstrates that bidirectional refinement is a genuinely useful decoding primitive for diffusion language models, not just a theoretical property. The draft-then-refine paradigm offers practitioners a simple, training-free inference-time optimization that can simultaneously improve accuracy and reduce latency, challenging the assumption that DLMs must be adapted to purely left-to-right generation.
Technical Details
- Flash-Flash configuration: The same LLaDA2.1-Flash model serves as both drafter (block-autoregressive generation) and refiner (bidirectional diffusion refinement), testing whether a model can improve its own autoregressive output through global refinement
- Mini-Flash configuration: Inspired by speculative decoding, LLaDA2.1-Mini drafts a full response and LLaDA2.1-Flash revises it as an editable initialization, enabling heterogeneous model cascades without additional training
- Benchmark results: Flash-Flash achieves GSM8K-384: 0.899 (up from 0.848), MBPP-384: 0.693 (up from 0.545); Mini-Flash achieves MATH-384: 0.294 vs Flash's 0.300 at 2.17x speedup
- Ablation findings: Refinement from fully masked spans performs poorly; completed drafts are essential; full global refinement provides clear GSM8K gains; local refinement captures much of the MBPP and MATH improvement
- Latency-window-matched controls confirm gains persist after targeted tuning of block-autoregressive decoding, ruling out trivial speed-accuracy tradeoffs
Industry Insight
- The draft-then-refine pattern offers an immediate, training-free optimization for any deployed diffusion language model, potentially improving both quality and throughput without retraining
- Speculative correction between model sizes (Mini drafting, Flash refining) provides a practical recipe for building fast DLM inference pipelines that approach full-model quality at significantly reduced latency
- Results support a Pareto-frontier interpretation rather than uniform quality matching, suggesting practitioners should carefully calibrate draft-refine configurations for their specific quality-latency requirements rather than assuming heterogeneous cascades will match the larger model
Disclaimer: The above content is generated by AI and is for reference only.