Rolling With Resistance: Preference-Optimized LLM Counselors Can Trade Goal Persistence for Relational Attunement in Motivational Interviewing
The paper introduces a two-axis evaluation framework (Goal Persistence and Relational Attunement) for assessing LLM counselors in Motivational Interviewing, based on the MITI coding system Using Direct Preference Optimization on the AnnoMI corpus, the study shows that penalizing confrontation reliably reduces goal persistence below parity across all tested models and seed runs Penalizing capitulation has no effect because the base models rarely exhibit capitulation on-policy, making the trade-of
Analysis
TL;DR
- The paper introduces a two-axis evaluation framework (Goal Persistence and Relational Attunement) for assessing LLM counselors in Motivational Interviewing, based on the MITI coding system
- Using Direct Preference Optimization on the AnnoMI corpus, the study shows that penalizing confrontation reliably reduces goal persistence below parity across all tested models and seed runs
- Penalizing capitulation has no effect because the base models rarely exhibit capitulation on-policy, making the trade-off gated by each model's failure profile
- A prompt-only control achieves attunement gains without the goal-persistence cost, indicating the trade-off originates from the optimization process itself rather than from attunement behavior
- The firewall evaluation protocol uses disjoint model families for generation, labeling, and judging to ensure unbiased assessment
Why It Matters
This research directly addresses a critical challenge in deploying LLMs for therapeutic and counseling applications: optimizing for one dimension of clinical quality can inadvertently degrade another. For AI practitioners building conversational agents in sensitive domains, the findings serve as a cautionary demonstration that preference optimization can produce unintended behavioral trade-offs that are not immediately obvious.
Technical Details
- The study uses the Motivational Interviewing Treatment Integrity (MITI) code to define two evaluation axes: Goal Persistence (maintaining the change agenda) and Relational Attunement (responding empathetically to client resistance)
- Preference optimization data is constructed from the expert-annotated AnnoMI corpus using topic-disjoint DPO pairs with on-policy negatives, where preference sets differ only in which failure mode (capitulation vs. confrontation) is rejected
- Evaluation employs a blind pairwise win-rate protocol with a firewall ensuring disjoint model families handle generation, labeling, and judging; the automatic judge is validated against expert labels and rechecked by trained human coders
- Three aligned instruction models spanning Qwen and Llama families are tested, with results showing base-dependent attunement gains but consistently negative goal-persistence effects when confrontation is penalized
Industry Insight
- When fine-tuning LLMs for clinical or counseling applications, practitioners should monitor for unintended trade-offs between competing quality dimensions, as optimizing one axis can silently degrade another
- Prompt-only interventions should be evaluated alongside preference optimization, as they may achieve similar attunement gains without the behavioral costs observed in DPO-trained models
- The firewall evaluation methodology presented here offers a robust template for reducing evaluation bias in AI safety and alignment research
Disclaimer: The above content is generated by AI and is for reference only.