Diff Mining: Logit Differences Reveal Finetuning Objectives
Diff Mining is a framework that identifies finetuning objectives by comparing output logits of finetuned and base models, requiring only black-box logit access rather than internal model weights The method uses a two-stage pipeline: extracting per-context logit differences on a reference corpus, then aggregating signals via Top-K frequency or Non-negative Matrix Factorization (NMF) to produce interpretable token fingerprints Diff Mining outperforms state-of-the-art model diffing methods in finet
Analysis
TL;DR
- Diff Mining is a framework that identifies finetuning objectives by comparing output logits of finetuned and base models, requiring only black-box logit access rather than internal model weights
- The method uses a two-stage pipeline: extracting per-context logit differences on a reference corpus, then aggregating signals via Top-K frequency or Non-negative Matrix Factorization (NMF) to produce interpretable token fingerprints
- Diff Mining outperforms state-of-the-art model diffing methods in finetune domain detection, both in token identification accuracy and downstream interpretability agent performance
- The framework successfully identifies over one-third of injected biases in models without any targeted probing, demonstrating utility for auditing unwanted finetuning behaviors
- The approach generalizes across domains, surfacing salient tokens even on text unrelated to the finetuning domain, making it a scalable tool for large model analysis
Why It Matters
As finetuning becomes the dominant paradigm for adapting language models, the opacity of what behaviors emerge during this process poses significant risks for safety, alignment, and accountability. Diff Mining offers a practical, scalable auditing mechanism that requires only logit access—making it applicable to proprietary and large-scale models where internal parameters are inaccessible. This has direct implications for researchers and practitioners seeking to understand, verify, and regulate finetuned model behavior.
Technical Details
- Two-stage modular architecture: Stage one computes per-context logit differences between the finetuned and base model across a reference corpus; stage two aggregates these differences into an interpretable token set representing the finetuning objective
- Two aggregation strategies: (i) Top-K frequency method, which selects the most frequently amplified tokens across contexts; (ii) NMF-based approach, which disentangles multiple finetuning objectives into distinct token clusters by factorizing the logit difference matrix
- Black-box compatibility: Unlike existing model diffing methods that require access to internal activations or weights, Diff Mining operates solely on output logits, enabling application to large and proprietary models
- Empirical validation: Evaluated on finetune domain detection (outperforming SOTA methods in both token identification and downstream interpretability agent tasks) and injected bias detection (recovering over one-third of biases without targeted probing)
Industry Insight
- The framework enables third-party auditing of finetuned models without requiring model internals, which could become a standard tool for regulatory compliance and safety evaluation as finetuned models are deployed in high-stakes domains
- The NMF-based disentanglement of multiple finetuning objectives into token clusters opens the door to decomposing complex, multi-objective finetuning pipelines into interpretable components—a capability increasingly valuable as models are trained with layered reward signals and preference data
- The ability to detect unwanted behaviors from logit differences alone suggests that lightweight monitoring tools could be integrated into deployment pipelines to flag anomalous finetuning effects before models reach production
Disclaimer: The above content is generated by AI and is for reference only.