Interpretable Symptom Vectors for Depression in a Large Language Model
Researchers used mechanistic interpretability on Gemma-3-27B-PT to identify how depressive symptoms are represented in internal model activations Symptom groups geometrically separate most prominently at layer 21 across multiple distance metrics Semantic Projection onto Symptom Vectors preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes A single depression vector in Layer 21 achieves AUC = 0.789 in separating depressive from non-depressive text The depression v
Analysis
TL;DR
- Researchers used mechanistic interpretability on Gemma-3-27B-PT to identify how depressive symptoms are represented in internal model activations
- Symptom groups geometrically separate most prominently at layer 21 across multiple distance metrics
- Semantic Projection onto Symptom Vectors preserved clinician-annotated rank ordering across mood, somatic, and suicidality axes
- A single depression vector in Layer 21 achieves AUC = 0.789 in separating depressive from non-depressive text
- The depression vector can serve as an emotional valence gate to restrict symptom projection to relevant depressive speech
Why It Matters
This work bridges mechanistic interpretability with clinical mental health assessment, demonstrating that LLMs encode clinically meaningful symptom structures in their internal activations. For AI practitioners and researchers, it provides a blueprint for building interpretable, trust-worthy AI tools in sensitive healthcare domains where black-box predictions face significant adoption barriers.
Technical Details
- Model: Gemma-3-27B-PT, analyzed via mechanistic interpretability techniques on the residual stream
- Layer 21 Discovery: Symptom groups showed the strongest geometric separation at layer 21 across multiple distance metrics, suggesting this layer encodes clinically structured depression-related representations
- Semantic Projection: Symptom Vectors were constructed from activations elicited by symptom descriptions drawn from validated clinical instruments, then used to project held-out naturalistic text onto per-symptom coefficient spaces
- Validation: Projected coefficients preserved clinician-annotated rank ordering across three axes—mood, somatic, and suicidality—demonstrating alignment with clinical judgment
- Depression Vector & Valence Gate: A single depression vector in Layer 21 separates depressive from non-depressive text with AUC = 0.789; this vector functions as an emotional valence gate to ensure symptom projection is applied only to depressive speech
Industry Insight
- Mechanistic interpretability can unlock clinically aligned representations in off-the-shelf LLMs without fine-tuning, reducing the cost and regulatory burden of building healthcare AI tools
- The "valence gate" concept—using a binary detection vector to condition downstream analysis—offers a generalizable pattern for ensuring AI tools operate only within their intended scope, critical for clinical safety
- As regulatory frameworks for AI in healthcare tighten, methods that provide interpretable, clinician-validated signals from internal activations will become essential for trust and compliance
Disclaimer: The above content is generated by AI and is for reference only.