DWT-Fusion: A Signal-Based Framework for Training-Free LLM-Generated Text Detection
DWT-Fusion introduces a training-free framework for detecting LLM-generated text using discrete wavelet analysis of token-level log-probability sequences. The framework leverages wavelet-based multiresolution signal representations to capture localized probability dynamics, improving detection performance. Four voting variants (equal-weight hard/soft, calibration-weighted hard/soft) are evaluated to combine wavelet configurations without supervised training. Best single wavelet configurations ac
Analysis
TL;DR
- DWT-Fusion introduces a training-free framework for detecting LLM-generated text using discrete wavelet analysis of token-level log-probability sequences.
- The framework leverages wavelet-based multiresolution signal representations to capture localized probability dynamics, improving detection performance.
- Four voting variants (equal-weight hard/soft, calibration-weighted hard/soft) are evaluated to combine wavelet configurations without supervised training.
- Best single wavelet configurations achieve AUROC of 0.9872 (HC3), 0.8185 (M4), and 0.7138 (MAGE), with calibration-weighted voting further improving AUROC to 0.9919, 0.8477, and 0.7471, respectively.
- The method demonstrates effective and interpretable signals for training-free LLM-generated text detection across diverse datasets and proxy models.
Why It Matters
This work addresses a critical challenge in AI: detecting LLM-generated text without relying on training data, which is essential for generalization across unseen generators and domains. By exploiting local and multiscale variations in token-level predictability via wavelet analysis, DWT-Fusion offers a novel, interpretable alternative to global statistical measures, advancing the state-of-the-art in zero-shot detection. The framework’s strong performance on multiple datasets and its training-free nature make it highly relevant for practitioners seeking robust, deployable solutions in dynamic AI environments.
Technical Details
- Core Method: DWT-Fusion uses discrete wavelet transform (DWT) to analyze token-level log-probability sequences generated by a proxy causal language model (e.g., GPT-Neo-2.7B, GPT-J-6B, Falcon-7B, LLaMA-3-8B).
- Multiresolution Analysis: Wavelet-based multiresolution representations capture localized probability dynamics, enabling detection of subtle patterns in LLM-generated text that global measures may miss.
- Voting Variants: Four training-free voting strategies are evaluated: equal-weight hard voting, equal-weight soft voting, calibration-weighted hard voting, and calibration-weighted soft voting, which combine multiple wavelet configurations without supervised meta-classifier training.
- Evaluation Datasets: HC3, M4, and MAGE are used to assess generalization across diverse text domains and generators.
- Performance Metrics: AUROC is the primary metric, with best single wavelet configurations achieving 0.9872 (HC3), 0.8185 (M4), and 0.7138 (MAGE), and calibration-weighted voting improving these to 0.9919, 0.8477, and 0.7471, respectively.
Industry Insight
- Generalization Potential: DWT-Fusion’s training-free approach reduces dependency on labeled data, making it adaptable to emerging LLMs and unseen text domains, a critical advantage for real-world deployment.
- Interpretability: The use of wavelet analysis provides interpretable signals, enabling practitioners to understand detection rationale and refine models based on localized probability dynamics.
- Scalability: The framework’s lightweight design (no supervised training) allows efficient integration into existing systems, supporting scalable detection solutions for content moderation, plagiarism detection, and AI safety applications.
Disclaimer: The above content is generated by AI and is for reference only.