A Transdiagnostic Space of Disorder Like Phenotypes in Reinforcement Learning Agents
The study introduces a transdiagnostic framework for modeling seven psychological disorders in reinforcement learning agents using dose-controllable manipulation of cognitive appraisal signals. Extensive testing across 1,000+ runs reveals graded, monotone dose-responses for all disorders, with self-organization into a two-dimensional affective space where mania mirrors anxiety. Treatment efficacy varies by disorder type: reward distortion disorders remit when knobs are removed, while avoidance d
Analysis
TL;DR
- The study introduces a transdiagnostic framework for modeling seven psychological disorders in reinforcement learning agents using dose-controllable manipulation of cognitive appraisal signals.
- Extensive testing across 1,000+ runs reveals graded, monotone dose-responses for all disorders, with self-organization into a two-dimensional affective space where mania mirrors anxiety.
- Treatment efficacy varies by disorder type: reward distortion disorders remit when knobs are removed, while avoidance disorders require graded exposure curricula.
- The framework demonstrates generalizability, successfully transferring disorder models to 3D pixel environments with standard convolutional agents, independent of specific architectures like PPO's appraisal critic.
Why It Matters
This research provides a rigorous, scalable testbed for computational psychiatry, allowing researchers to simulate and study complex mental health conditions in controlled AI environments. By establishing a unified parameterization of affective phenotypes, it offers actionable insights into how different cognitive distortions interact and how they might be treated, bridging the gap between theoretical models and empirical observation in both AI and human psychology.
Technical Details
- Methodology: Utilizes an appraisal-guided Proximal Policy Optimization (PPO) agent where seven disorders (anxiety, mania, OCD, depression, impulsivity, addiction, PTSD) are induced via single-knob manipulation of cognitive appraisal weights.
- Validation: Conducted over 1,000 runs with 10 seeds and four controls, ensuring statistical significance through 95% confidence intervals and preregistered behavioral assays mapped to recognized psychological paradigms.
- Key Findings: Identified non-additive interactions between simultaneous knobs leading to comorbidity predictions; discovered distinct recovery pathways for reward distortion vs. avoidance disorders.
- Generalization: Validated transferability of three disorder knobs (depression, addiction, anxiety) to a 3D MiniWorld environment using a standard convolutional agent without an appraisal critic, confirming domain independence.
Industry Insight
- Computational Psychiatry: Researchers can leverage this framework to generate testable hypotheses about mental health mechanisms and treatment efficacy before moving to clinical trials.
- AI Safety and Alignment: Understanding how specific parameter manipulations lead to pathological behaviors helps in designing more robust and emotionally stable AI agents, particularly those intended for human interaction.
- Interdisciplinary Collaboration: This work highlights the potential for AI simulations to inform psychological theory, encouraging deeper collaboration between computer scientists and mental health professionals to refine diagnostic models.
Disclaimer: The above content is generated by AI and is for reference only.