PAC-Private Autoregressive Generation: Calibrating Noise to Ensemble Disagreement
Extends PAC (Private Autocorrelation) privacy from classification to autoregressive text generation, a first in the literature Uses 128 overlapping "worlds" from the private corpus, each training a separate adapter over a frozen public model, with noise calibrated to ensemble disagreement rather than fixed per-token budgets Introduces coupled decoding to prevent greedy degeneration while preserving privacy accounting under adaptive self-generated contexts Achieves 74% retention of fine-tuning ga
Analysis
TL;DR
- Extends PAC (Private Autocorrelation) privacy from classification to autoregressive text generation, a first in the literature
- Uses 128 overlapping "worlds" from the private corpus, each training a separate adapter over a frozen public model, with noise calibrated to ensemble disagreement rather than fixed per-token budgets
- Introduces coupled decoding to prevent greedy degeneration while preserving privacy accounting under adaptive self-generated contexts
- Achieves 74% retention of fine-tuning gain at a per-token budget of 2^-32 on WikiText-103 with GPT-2-small, with membership-inference success bounded at 51.08% after 10^6 tokens
- Outperforms PMixED significantly, retaining 98% of non-private headroom versus at most 56% across 10^2 to 10^6 tokens under matched membership-inference bounds
Why It Matters
This work addresses a critical gap in privacy-preserving AI: while private inference methods exist for classification, autoregressive generation—where models serve as APIs producing sequential outputs—remains largely unprotected. For practitioners deploying fine-tuned language models on sensitive corpora, this framework offers a practical path to limit privacy leakage through generated text without catastrophic utility loss, making it directly relevant to any organization serving private-data-adapted LLMs via API.
Technical Details
- World construction: 128 overlapping worlds are built from the private corpus, with each record appearing in exactly 64 worlds; one LoRA adapter is trained per world over a frozen public model (GPT-2-small), and the realized world constitutes the secret
- Noise calibration mechanism: At each token step, the public model defines a candidate set, the 128 world-adapters vote, and posterior-weighted disagreement across worlds determines the magnitude of PAC noise added; unanimous predictions require zero calibration noise
- Theoretical guarantee: Proves that mutual information between the secret S and generated output Y_{1:T} is bounded by I(S;H_T) ≤ bT, where H_T represents the hidden state at step T
- Coupled decoding: A novel decoding strategy that preserves privacy accounting while avoiding the degeneration that occurs with naive greedy decoding under noise injection
- Empirical evaluation: Tested on WikiText-103 with GPT-2-small; posterior-entropy estimates of actual leakage are ~17% of the charged privacy budget, indicating substantial margin in the theoretical bounds
Industry Insight
- The finding that inference privacy does not equal content protection—memorized canaries are emitted at the same rate even when membership advantage is near zero—should caution practitioners against over-relying on membership-inference metrics as the sole privacy guarantee; complementary content-level safeguards remain necessary
- The dramatic performance gap versus PMixED (98% vs 56% headroom retention) suggests that fixed-per-token privacy accounting is fundamentally ill-suited for long-horizon autoregressive tasks, and adaptive noise calibration should become a standard design principle
- The adapter-per-world architecture, while elegant, scales linearly with the number of worlds; for production deployment with large corpora, researchers should explore compressed world representations or sparse ensemble selection to reduce computational overhead without sacrificing privacy guarantees
Disclaimer: The above content is generated by AI and is for reference only.