A Single Suffix to Break Them All: Basin-Aware Jailbreaks for Merged Model Families
Model merging of individually safety-aligned models can still produce unsafe outputs due to risks rooted in the pretrained foundation backbone, challenging the assumption that merging preserves safety The authors introduce Basin-Aware Jailbreak (BAJ), a novel attack that generates adversarial suffixes transferable across merged model families sharing the same pretrained backbone BAJ formulates jailbreak generation as a min-max optimization over the merging space, enabling attacks without knowled
Analysis
TL;DR
- Model merging of individually safety-aligned models can still produce unsafe outputs due to risks rooted in the pretrained foundation backbone, challenging the assumption that merging preserves safety
- The authors introduce Basin-Aware Jailbreak (BAJ), a novel attack that generates adversarial suffixes transferable across merged model families sharing the same pretrained backbone
- BAJ formulates jailbreak generation as a min-max optimization over the merging space, enabling attacks without knowledge of exact merging coefficients or constituent checkpoints
- Experiments across diverse backbones and merging settings demonstrate consistently high transfer success rates, with effectiveness persisting under existing defenses
Why It Matters
This work exposes a critical blind spot in the model merging ecosystem: safety alignment of individual fine-tuned models does not guarantee safety in merged outputs, as the underlying pretrained foundation can reintroduce vulnerabilities. For AI practitioners deploying merged models in production, this represents a novel attack surface that existing safety evaluations may fail to detect, necessitating new benchmarking and defense strategies tailored to merged model families.
Technical Details
- Threat model: The attacker constructs jailbreak prompts that generalize across merged models sharing a common pretrained backbone, without access to exact merging coefficients or individual constituent checkpoints — a realistic and challenging setting
- BAJ method: Jailbreak generation is formulated as a min-max optimization problem over the merging space, producing adversarial suffixes that exploit the shared foundation model's vulnerability basin rather than individual fine-tuned parameters
- Key insight: Prior work attributed merging risks to unsafe constituent models; this work demonstrates that even when all constituents are individually safety-aligned, the pretrained foundation's inherent vulnerabilities can be activated through merging
- Evaluation: Experiments conducted across diverse pretrained backbones and merging configurations, showing high transfer success rates of adversarial suffixes across merged model families and resilience against existing defensive measures
Industry Insight
- Organizations adopting model merging as a cost-effective fine-tuning strategy must reassess their safety evaluation pipelines to account for foundation-level vulnerabilities that merging can reactivate, not just constituent model risks
- The transferability of BAJ-style attacks across merged families suggests that adversarial suffixes discovered on one merged model may compromise an entire family of models sharing a backbone, amplifying the blast radius of a single successful jailbreak
- Defense research should prioritize robustness over the merging space itself — potentially through merging-aware alignment techniques or foundation model hardening — rather than treating each merged variant as an independent safety problem
Disclaimer: The above content is generated by AI and is for reference only.