Larger Context Window, Fewer Overcorrections: Optimizing Prompts and Batching for Minimal-Edit Grammatical Error Correction
Minimal-edit GEC is a persistent challenge for zero/few-shot LLMs, which systematically overcorrect and degrade F0.5 by rewriting well-formed text spans Three prompt-based advances close the gap to fine-tuned models: taxonomy-based instructions with comprehensive grammatical error rules, batching multiple sentences as a regularizer against overcorrection, and LLM-assisted prompt optimization Batching multiple uncorrected sentences into a single context acts as a targeted regularizer, hypothesize
Analysis
TL;DR
- Minimal-edit GEC is a persistent challenge for zero/few-shot LLMs, which systematically overcorrect and degrade F0.5 by rewriting well-formed text spans
- Three prompt-based advances close the gap to fine-tuned models: taxonomy-based instructions with comprehensive grammatical error rules, batching multiple sentences as a regularizer against overcorrection, and LLM-assisted prompt optimization
- Batching multiple uncorrected sentences into a single context acts as a targeted regularizer, hypothesized to arise from an attention dilution effect induced by the bounded capacity of self-attention scores
- The approach achieves F0.5=78.32 on BEA-2019 test set using Gemini 3.1-Pro, establishing a new prompt-based SOTA and narrowing the gap to fine-tuned SOTA to just 0.38 points
- Code, prompts, and outputs are publicly available, lowering the infrastructure barrier for high-quality GEC
Why It Matters
This work demonstrates that carefully engineered prompting strategies can nearly match fine-tuned model performance on a challenging NLP task, offering a cost-effective alternative to expensive fine-tuning pipelines. For AI practitioners working on language quality tools, it provides a practical framework for reducing overcorrection—a pervasive failure mode in LLM-based text editing—without requiring dedicated training infrastructure.
Technical Details
- Taxonomy-based instructions: The authors introduce a comprehensive list of grammatical error rules that bound the LLM's scope of correctable edits, aligning model behavior with the F0.5 evaluation metric. This approach benefits the strongest models while remaining model-dependent overall.
- Batching as regularization: Multiple uncorrected sentences are batched into a single input context, which systematically reduces the edit rate across diverse LLM families. The hypothesized mechanism is an attention dilution effect caused by the bounded capacity of self-attention scores.
- LLM-assisted Prompt Optimization: Gemini 3.1-Pro is used to iteratively refine the prompt instructions, automating what would otherwise be manual prompt engineering.
- Benchmark performance: Achieves F0.5=78.32 on the BEA-2019 test set, establishing a new prompt-based state-of-the-art and closing the gap to the fine-tuned single-model SOTA (Staruch et al., 2025) to just 0.38 points.
- Minimal-edit constraint: The core technical challenge addressed is preventing LLMs from rewriting well-formed spans—a systematic overcorrection behavior that degrades precision without improving recall.
Industry Insight
- The attention dilution hypothesis from batching suggests a generalizable technique for controlling LLM aggressiveness in editing tasks, potentially applicable beyond GEC to code correction, summarization, and other text transformation pipelines.
- The near-parity between prompt-based and fine-tuned approaches (0.38-point gap) signals that for many production GEC use cases, organizations can skip expensive fine-tuning and invest in prompt engineering instead, significantly reducing infrastructure costs and time-to-deployment.
- As LLM context windows continue to grow, the batching strategy becomes increasingly viable—practitioners should experiment with larger batch sizes to further suppress overcorrection in their own text editing applications.
Disclaimer: The above content is generated by AI and is for reference only.