Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal
The paper introduces "narrow-boundary safety," arguing that safety alignment should be deployment-specific rather than topic-level, as different applications need different refusal boundaries within the same topic An offline self-generated framework combining controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs is proposed for training and evaluation On Qwen3-8B with political persuasion tasks, escalation-based training increased target-domain
Analysis
TL;DR
- The paper introduces "narrow-boundary safety," arguing that safety alignment should be deployment-specific rather than topic-level, as different applications need different refusal boundaries within the same topic
- An offline self-generated framework combining controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs is proposed for training and evaluation
- On Qwen3-8B with political persuasion tasks, escalation-based training increased target-domain refusal from 9.47% to 84.75% while reducing unsafe responses across benchmarks from 26.26% to 0.14%
- The approach significantly increases XSTest over-refusal (2.00% to 74.00%), highlighting a critical safety-usability trade-off that data composition can control
- Using verified target-model responses instead of external responses reduces over-refusal from 15.20% to 5.20%, and boundary-pair data reduces comply-side over-refusal from 32.94% to 4.16%
Why It Matters
This work challenges the one-size-fits-all approach to LLM safety alignment by demonstrating that different deployments (e.g., civics tutor vs. public-sector assistant) require distinct refusal boundaries even within the same topic domain. For AI practitioners, it provides a practical self-distillation framework that can be adapted to specific deployment contexts, while also exposing the significant over-refusal risks that come with targeted safety tuning.
Technical Details
- Narrow-boundary safety formulation: Safety is reframed from topic-level (is this subject harmful?) to boundary-level (what specific behaviors within a topic should be refused), enabling deployment-specific alignment
- Self-generated training framework: Combines controlled topic generation, coverage repair (escalating retries reduce unaccepted refusal traces from 19.88% to 0.20%), in-distribution compensation data, and harmful-benign pairs
- Qwen3-8B experiments on political persuasion: Target-domain refusal improved from 9.47% to 84.75%; mean unsafe-response rate across three broader harmfulness benchmarks dropped from 26.26% to 0.14%
- Over-refusal trade-offs: XSTest over-refusal increased from 2.00% to 74.00% with escalation training; replacing external responses with verified target-model responses reduced over-refusal from 15.20% to 5.20%
- Boundary-pair data effects: Reduced comply-side over-refusal on held-out pairs from 32.94% to 4.16%, while harmful-side refusal decreased only marginally from 91.88% to 87.72%
Industry Insight
- Safety evaluation must measure both sides of the refusal boundary—compliance on benign prompts and refusal on harmful ones—since optimizing for one often degrades the other
- Data composition is the primary lever for controlling the safety-usability trade-off; practitioners should invest in high-quality, deployment-matched refusal data rather than relying on generic safety benchmarks
- The dramatic over-refusal increase (up to 74%) suggests that current safety alignment methods may be too blunt for production deployments requiring nuanced, context-aware behavior, calling for more refined evaluation metrics beyond standard benchmarks like XSTest
Disclaimer: The above content is generated by AI and is for reference only.