Selection, Recombination, or a Fresh Solve? A Candidate-Free Control for Single-Pass Test-Time Aggregation
Introduces a candidate-free control for single-pass test-time aggregation, comparing candidate conditioning against a fresh solve under identical token budgets On AIME-2025 and HMMT-2025 with Qwen3-4B, candidate conditioning boosts accuracy by +0.290 when multiple candidates are correct but reduces it by -0.123 when all candidates are wrong The one-correct candidate regime remains unresolved; the all-wrong recovery interpretation reverses under conservative correction for the adaptive two-benchm
Analysis
TL;DR
- Introduces a candidate-free control for single-pass test-time aggregation, comparing candidate conditioning against a fresh solve under identical token budgets
- On AIME-2025 and HMMT-2025 with Qwen3-4B, candidate conditioning boosts accuracy by +0.290 when multiple candidates are correct but reduces it by -0.123 when all candidates are wrong
- The one-correct candidate regime remains unresolved; the all-wrong recovery interpretation reverses under conservative correction for the adaptive two-benchmark procedure
- Structured interventions show explicit answer fields causally steer outputs, while masking yields no measurable accuracy gain
- Evidence is scoped to one model family (Qwen3-4B), two math benchmarks, first-answer-truncated candidates, and single-pass aggregation prompts
Why It Matters
This work directly addresses a core tension in test-time compute scaling: whether conditioning an aggregation call on generated candidates provides genuine reasoning value or merely wastes tokens on redundant context. For practitioners building test-time reasoning pipelines, the findings suggest that candidate pools are a double-edged sword—helpful when diversity yields correct answers but harmful when all candidates share the same failure mode. The candidate-free control methodology also sets a rigorous benchmark for evaluating future aggregation strategies.
Technical Details
- Candidate-free control: The paper introduces a baseline where the aggregation model solves the problem afresh without any candidate context, matched to the same maximum output-token allowance as the candidate-conditioned variant
- Stratification by correct candidate count: Results are split into three regimes—c0 (all wrong), c1 (one correct), and c2+ (multiple correct)—revealing asymmetric effects of candidate conditioning
- Benchmarks and model: Evaluated on AIME-2025 and HMMT-2025 mathematics competitions using Qwen3-4B, with first-answer-truncated candidate fragments and single-pass prompted aggregation
- Structured intervention: Explicit answer fields were shown to causally steer model outputs toward their values; masking interventions produced no measurable accuracy improvement, and format equivalence was not established
- Statistical correction: Conclusions for c2+ and c0 survive a conservative correction for the adaptive two-benchmark procedure, strengthening confidence in those findings
Industry Insight
- Test-time aggregation strategies should be regime-aware: invest in candidate generation and conditioning when expecting multiple plausible paths, but consider fresh solves when candidates are likely uniformly incorrect
- The negative effect of all-wrong candidate conditioning (-0.123) suggests that polluted candidate context can actively mislead aggregation models, implying that candidate filtering or quality gates may be essential before aggregation
- Explicit output formatting (e.g., structured answer fields) is a leveraged causal intervention worth incorporating into pipeline design, while naive masking approaches should be abandoned in favor of format-aware prompting
Disclaimer: The above content is generated by AI and is for reference only.