Compliance, Capability, and Conflict: Benchmarking Multimodal LLMs under System Messages
Introduces VSysBench, a benchmark built on MMVet-v2 that evaluates how Multimodal Large Language Models (MLLMs) adhere to system messages across 5 main categories and 22 sub-categories of constraints Proposes two evaluation metrics: Joint Satisfaction Rate (JSR) measuring simultaneous constraint compliance and answer correctness, and Cross-Constraint Sensitivity (CCS) assessing how constraints interact System messages substantially erode base task accuracy across 16 evaluated MLLMs, revealing a
Analysis
TL;DR
- Introduces VSysBench, a benchmark built on MMVet-v2 that evaluates how Multimodal Large Language Models (MLLMs) adhere to system messages across 5 main categories and 22 sub-categories of constraints
- Proposes two evaluation metrics: Joint Satisfaction Rate (JSR) measuring simultaneous constraint compliance and answer correctness, and Cross-Constraint Sensitivity (CCS) assessing how constraints interact
- System messages substantially erode base task accuracy across 16 evaluated MLLMs, revealing a significant compliance-capability tradeoff
- Open-weight models show compliance collapse under user conflict, while top proprietary models maintain stable compliance
- Vision-grounded constraints are consistently the hardest category for every model tested
Why It Matters
This research addresses a critical gap in MLLM evaluation: existing benchmarks fail to measure how well models follow system-level instructions in multimodal contexts, despite production deployments increasingly relying on them. The findings have direct implications for practitioners deploying MLLMs in real-world scenarios where system messages must coexist with strong vision-language capabilities, and where user conflicts may arise between system directives and user prompts.
Technical Details
- VSysBench is constructed on top of MMVet-v2, organizing constraints into 5 main categories and 22 sub-categories spanning from textual directives in visual contexts to fully vision-grounded constraints, each paired with a misaligned counterpart to stress-test the instructional hierarchy
- Two novel metrics are introduced: Joint Satisfaction Rate (JSR) scores responses jointly along constraint compliance and answer correctness axes, while Cross-Constraint Sensitivity (CCS) measures how the presence of one constraint affects compliance with another
- The benchmark evaluates 16 MLLMs (both open-weight and proprietary), testing them under conditions where system messages may conflict with user instructions
- Constraints range across textual directives in visual contexts to fully vision-grounded ones, with misaligned counterparts designed to create tension between system and user instructions
Industry Insight
- Deployment pipelines for MLLMs must account for the significant accuracy degradation caused by system messages; practitioners should expect and plan for capability tradeoffs when implementing behavioral constraints
- Open-weight models remain vulnerable to instruction hierarchy breakdown under user conflict, making proprietary models a safer choice for production environments where adversarial or conflicting user inputs are likely
- Vision-grounded constraints represent the hardest evaluation category across all models, suggesting that future research should prioritize improving how models integrate visual information with system-level behavioral instructions
Disclaimer: The above content is generated by AI and is for reference only.