Probe Generalization as Subspace Selection for OOD Deception Detection
Linear probes for detecting deception in LLM activations can generalize across domains by projecting inputs onto a curated subset of principal components from training data Using an LLM judge to score PCs on transferability closes 78% of the performance gap on Insider Trading Report and 25% on Sandbagging datasets Source-specific surface features differ from transferable directions, which encode more abstract conceptual contrasts expressible in natural language OOD robustness of probes is fundam
Analysis
TL;DR
- Linear probes for detecting deception in LLM activations can generalize across domains by projecting inputs onto a curated subset of principal components from training data
- Using an LLM judge to score PCs on transferability closes 78% of the performance gap on Insider Trading Report and 25% on Sandbagging datasets
- Source-specific surface features differ from transferable directions, which encode more abstract conceptual contrasts expressible in natural language
- OOD robustness of probes is fundamentally determined by subspace selection rather than probe architecture alone
- The approach enables near-oracle performance without retraining probes on target distribution data
Why It Matters
This research addresses a critical bottleneck in mechanistic interpretability: linear probes often fail to generalize across domains, limiting their practical utility for safety monitoring. By demonstrating that subspace selection can recover most transferable signal, the work provides a scalable pathway for deploying deception detection probes across diverse real-world scenarios without expensive per-domain retraining.
Technical Details
- Model & Probes: Evaluated on Llama-3.1-8B-Instruct using linear probes trained on deception detection tasks and tested across 3 held-out OOD datasets (Insider Trading Report, Sandbagging, and a third dataset)
- PCA-based Subspace Selection: Projected activation inputs onto principal components from the training distribution, then selected a small subset of transferable PCs rather than using all components
- LLM Judge Scoring: Used an LLM as a judge to evaluate each PC by examining its most and least activating examples, scoring whether they implied a transferable deception direction
- Performance Gains: Closed 78% of the baseline-to-oracle gap on Insider Trading Report and 25% on Sandbagging by probing only on highest-scoring PCs
- Interpretability Finding: Heavy-weighted source probe directions encoded surface-level features, while transferable directions captured abstract contrasts that natural language descriptions could articulate
Industry Insight
- Interpretability teams should prioritize subspace selection as a first step when deploying probes across domains, rather than assuming probe weights themselves generalize
- LLM-based automated scoring of activation directions offers a practical, scalable alternative to manual feature analysis for identifying transferable interpretability signals
- The finding that transferable directions encode abstract rather than surface features suggests that interpretability research should focus on discovering higher-level conceptual representations for robust cross-domain deployment
Disclaimer: The above content is generated by AI and is for reference only.