How Do Linear Probes Emerge? A Circuit-Tracing Framework with Concept-Targeted Attribution
Introduces Concept-Targeted Attribution (CTA), a framework that trains transcoder attribution graphs with respect to linear probe directions rather than next-token probabilities, enabling probe-specific circuit explanations. CTA yields probe-targeted graphs with strong predictive structure: graph-level features predict probe accuracy across four concept categories (ρ = 0.91, R² = 0.84), while local features identify sparse per-prompt classification drivers. Causal ablations demonstrate that prob
Analysis
TL;DR
- Introduces Concept-Targeted Attribution (CTA), a framework that trains transcoder attribution graphs with respect to linear probe directions rather than next-token probabilities, enabling probe-specific circuit explanations.
- CTA yields probe-targeted graphs with strong predictive structure: graph-level features predict probe accuracy across four concept categories (ρ = 0.91, R² = 0.84), while local features identify sparse per-prompt classification drivers.
- Causal ablations demonstrate that probe-targeted and logit-targeted graphs capture functionally distinct mechanisms—removing probe-relevant features reduces internal concept scores while preserving generated tokens, whereas removing logit-relevant features changes tokens in 92–100% of cases with near-zero effect on probe scores.
- The framework bridges behavioral probe accuracy and mechanistic circuit-level explanations, enabling detailed audits of internal concept representations including safety-critical ones.
Why It Matters
This work represents a significant methodological advance in mechanistic interpretability by decoupling the explanation of internal concept representations from output behavior, addressing a long-standing limitation where attribution methods were tied exclusively to token prediction. For AI practitioners and researchers, it provides a practical tool to audit whether models genuinely encode safety-critical or sensitive concepts internally, independent of whether those concepts surface in generated text.
Technical Details
- Concept-Targeted Attribution (CTA): Extends transcoder attribution graphs by training them against linear probe directions rather than logit targets, producing circuits that explain why an internal concept representation arises regardless of whether it is expressed in the generated token.
- Cross-Layer Transcoders: Used to construct probe-targeted graphs; graph-level features (e.g., structural properties) predict probe accuracy across four widely studied concept categories with high correlation (ρ = 0.91, R² = 0.84), while local features pinpoint sparse components responsible for per-prompt classification decisions.
- Causal Ablation Studies: Systematic removal experiments show functional dissociation—probe-relevant feature removal reduces internal concept scores with minimal impact on generated tokens, while logit-relevant feature removal alters generated tokens in 92–100% of cases with negligible effect on probe scores, confirming the two attribution targets capture distinct computational mechanisms.
- Four Concept Categories: Evaluated across widely studied concept types, demonstrating generalizability of the CTA framework beyond a single domain.
Industry Insight
- The dissociation between probe-targeted and logit-targeted mechanisms implies that standard interpretability audits based solely on output behavior may miss critical internal representations; practitioners should adopt concept-targeted approaches for safety-critical model audits.
- The high predictive power of graph-level features (R² = 0.84) suggests that structural properties of attribution circuits can serve as efficient proxies for probe performance, enabling scalable screening of model internals without per-prompt analysis.
- As linear probes remain a primary tool for probing model internals, CTA provides a principled pathway to move from correlation-based probe evaluation to causal, mechanistic understanding—essential for building trustworthy and auditable AI systems.
Disclaimer: The above content is generated by AI and is for reference only.