Revisiting Chain-of-Thought Reasoning under Limited Supervision: Semi-supervised Chain-of-Thought Learning
Introduces Semi-CoT, a framework that repurposes Chain-of-Thought reasoning traces from unlabeled data as semi-supervised learning signals rather than just inference-time prompts. Utilizes an entropy-based selection mechanism to filter pseudo-reasoning chains, achieving high precision (91.36% to 100%) in identifying reliable demonstrations. Extends the self-training paradigm for LLMs by integrating pseudo-supervision into the training loop, moving beyond standard prompt engineering. Demonstrates
Analysis
TL;DR
- Introduces Semi-CoT, a framework that repurposes Chain-of-Thought reasoning traces from unlabeled data as semi-supervised learning signals rather than just inference-time prompts.
- Utilizes an entropy-based selection mechanism to filter pseudo-reasoning chains, achieving high precision (91.36% to 100%) in identifying reliable demonstrations.
- Extends the self-training paradigm for LLMs by integrating pseudo-supervision into the training loop, moving beyond standard prompt engineering.
- Demonstrates mixed results across benchmarks, showing marginal gains on SVAMP and GSM8K but negative transfer on AQuA, highlighting challenges in generalization.
Why It Matters
This research addresses a critical bottleneck in AI development: the scarcity of high-quality labeled reasoning data. By demonstrating how unlabeled data can be leveraged to create reliable pseudo-supervision, it offers a scalable path to improving model reasoning capabilities without extensive manual annotation. For practitioners, it highlights the potential and pitfalls of semi-supervised learning in complex reasoning tasks, suggesting that better selection mechanisms are needed to fully unlock the value of unlabeled data.
Technical Details
- Framework: Semi-CoT generates multiple pseudo-CoT traces for each unlabeled question and employs an answer-level semantic entropy metric to evaluate consistency.
- Selection Mechanism: An "entropy gate" selects low-entropy reasoning chains as high-confidence pseudo-demonstrations for training, effectively filtering out noisy or incorrect reasoning paths.
- Benchmarks: Evaluated on mathematical reasoning datasets including AQuA, SVAMP, GSM8K, and MultiArith.
- Performance Metrics: Pseudo-answer precision ranged from 91.36% to 100%, indicating strong filtering capability, though final model performance gains were modest or negative depending on the dataset.
Industry Insight
- Data Efficiency: Organizations should prioritize collecting vast amounts of unlabeled domain-specific questions to build robust semi-supervised pipelines, reducing dependency on expensive labeled datasets.
- Quality Control: The negative transfer observed on AQuA suggests that naive application of pseudo-labeling can degrade performance; implementing rigorous entropy-based filtering or hybrid training strategies is essential.
- Future Research: There is significant opportunity to improve the "student training" phase of semi-supervised CoT, as the current pilot experiments indicate that selection alone is insufficient for consistent improvement across diverse reasoning tasks.
Disclaimer: The above content is generated by AI and is for reference only.