SynthSentry: Detecting Synthetic Data Contamination in Language Model Training Data
SynthSentry is a corpus-level, model-agnostic method for detecting synthetic data contamination in LLM training data without requiring access to the generating model, generation history, or synthetic labels The scoring mechanism combines three distributional divergence statistics: lexical diversity collapse, n-gram tail truncation, and perplexity variance across reference models A naive quantile-based threshold runs four times over the nominal false-positive budget; replacing it with covariance
Analysis
TL;DR
- SynthSentry is a corpus-level, model-agnostic method for detecting synthetic data contamination in LLM training data without requiring access to the generating model, generation history, or synthetic labels
- The scoring mechanism combines three distributional divergence statistics: lexical diversity collapse, n-gram tail truncation, and perplexity variance across reference models
- A naive quantile-based threshold runs four times over the nominal false-positive budget; replacing it with covariance shrinkage and a bootstrap threshold restores per-domain calibration
- The toolkit is released as a data-curation defense for pre-training screening, though results are limited to small-scale, English-language, batch-mode corpus screening with single-generation contamination sources
Why It Matters
As synthetic data becomes increasingly prevalent in training corpora, detecting contamination before training is critical to preventing model collapse and preserving factual accuracy. SynthSentry provides a practical, model-agnostic screening tool that shifts the paradigm from post-hoc diagnosis to proactive data curation, which is essential for any organization building or fine-tuning language models on mixed provenance data.
Technical Details
- Three-statistic scoring: The SynthSentry score measures distributional divergence across (1) lexical diversity collapse, (2) n-gram tail truncation, and (3) perplexity variance computed across multiple reference models
- Model-agnostic design: Requires no knowledge of the generating model, no generation history, and no synthetic labels, making it applicable to corpora of unknown provenance
- Threshold calibration: A naive quantile-based approach was found to produce false positives at four times the nominal budget; the authors replace this with covariance shrinkage and a bootstrap threshold to achieve proper per-domain calibration
- Evaluation protocol: Tested on corpora contaminated by small open-weight generators and an instruction-tuned open-weight model using a leave-one-generator-out protocol, with domain-stratified analysis on naturally repetitive human text (legal, clinical, source code)
- Limitations: Results are small-scale, English-only, batch-mode; contamination sources are single-generation or hand-authored rather than recursively generated, so findings generalize to synthetic contamination broadly but not specifically to recursion depth
Industry Insight
- Organizations ingesting third-party or web-scraped data should integrate pre-training contamination screening into their data curation pipelines; SynthSentry offers a ready-to-use toolkit for this purpose
- The finding that naive statistical thresholds dramatically overfit false positives underscores the importance of careful calibration—especially when screening domain-specific corpora like legal or clinical text where natural repetition is expected
- While the paper shows no contamination-driven accuracy deficit at their scale, the open question of whether pruning recovers lost performance means practitioners should treat SynthSentry as a risk-ranking tool rather than a definitive quality guarantee, and monitor over-pruning risks once removal exceeds true contamination fractions
Disclaimer: The above content is generated by AI and is for reference only.