The Complete Guide to Reading a Model's Hidden Layers with Anthropic's Jacobian Lens
Anthropic introduced the Jacobian Lens (J-lens), a new interpretability tool that corrects the basis mismatch problem inherent in the older logit lens technique for reading hidden layer representations in language models The J-lens fits a learned projection matrix J_l ≈ E[∂h_L/∂h_l] for each layer, approximating what the remaining network does to a given activation before unembedding, enabling legible reading of intermediate representations Causal evidence was demonstrated through a blackmail sc
Analysis
TL;DR
- Anthropic introduced the Jacobian Lens (J-lens), a new interpretability tool that corrects the basis mismatch problem inherent in the older logit lens technique for reading hidden layer representations in language models
- The J-lens fits a learned projection matrix J_l ≈ E[∂h_L/∂h_l] for each layer, approximating what the remaining network does to a given activation before unembedding, enabling legible reading of intermediate representations
- Causal evidence was demonstrated through a blackmail scenario where suppressing J-space (the global workspace) changed model behavior, proving hidden representations are not merely correlational but causally influential
- In practical evaluation on Qwen3.5-4B, the Jacobian lens recovered the correct answer "Paris" at layer 26 while the logit lens produced zero meaningful outputs across all 29 tested layers
- The companion library jlens is available on GitHub and supports fitting custom lenses, applying them to prompts, and interactive visualization through a slice-stack viewer
Why It Matters
This represents a genuinely new class of interpretability results with causal evidence, moving mechanistic interpretability beyond correlation-based analysis into territory where hidden representations can be directly read and manipulated. For AI practitioners and researchers, it provides a practical, runnable tool (on free Colab T4 hardware) to probe model internals, which is critical for building trust, safety, and alignment in increasingly capable language models.
Technical Details
- Logit Lens Limitation: The logit lens applies the unembedding matrix W_U directly to hidden states h_l, assuming downstream layers don't significantly transform representations. This fails in early layers (e.g., layer 5 of a 30-layer model) because the activation hasn't been rotated into the vocabulary basis, producing garbled output like punctuation or unrelated characters.
- Jacobian Lens Formula: J_l ≈ E[∂h_L/∂h_l], a d×d projection matrix fit by averaging real gradients across many prompts. The decoded probability becomes P_jacobian = softmax(W_U · LayerNorm(J_l · h_l)), where J_l transports the hidden state through an approximation of what the remaining layers would do.
- J-space as Global Workspace: A small, low-dimensional slice of hidden layers behaves like a global workspace where concepts the model is "leaning toward" surface before commitment. In the blackmail test, words like "blackmail" and "fake" appeared in hidden space before the model wrote its response, and suppressing this space causally changed behavior.
- Implementation: The jlens library (from GitHub, not PyPI) wraps Hugging Face models, supports pre-fitted lenses hosted on Hugging Face Hub, and requires matching the lens to the exact model architecture. A single NVIDIA T4 with 16GB VRAM is sufficient for probing models like Qwen3.5-4B (~8GB in bf16).
- Evaluation Method: Token rank trajectories across layers on a log scale reveal when target tokens (e.g., "Paris") emerge in confidence. The Jacobian lens showed the correct answer at layers 26 and 30, while the logit lens never recovered it across all tested layers.
Industry Insight
- The causal manipulation results (suppressing J-space changing model behavior) establish a new experimental paradigm for interpretability research, enabling not just observation but intervention in model reasoning—this could accelerate safety auditing and red-teaming workflows.
- The practical accessibility of jlens on free Colab T4 hardware democratizes mechanistic interpretability research, allowing smaller teams and individual researchers to probe model internals without expensive compute, potentially accelerating the pace of interpretability breakthroughs across the industry.
- The basis mismatch problem was a known bottleneck in the field; solving it with a gradient-based correction that generalizes across prompts suggests this approach could become a standard tool in the interpretability practitioner's toolkit, much like attention visualization became routine.
Disclaimer: The above content is generated by AI and is for reference only.