Exploring the adversarial robustness and safety alignment of multilingual multimodal large language models

The researchers conducted a systematic evaluation of adversarial robustness and multimodal safety for open-source multimodal large language models in 12 languages, targeting models that acquire multilingual capabilities through instruction tuning. Gradient-based attacks revealed transferable multilingual vulnerabilities: adversarial images optimized in one language can still induce failures in other languages. Safety performance depends on the model's ability to retrieve and interpret harmful instructions; for text input, a stronger language foundation makes it easier to elicit responses that enable misuse, whereas in typographic text embedded in images, English can be reliably recognized and followed, while non-English text is rarely parsed by the visual encoder. The paper calls the phenomenon whereby low-resource languages appear safer safety-by-failure, i.e., an artifact of failures in understanding and visual grounding rather than genuine alignment; Qwen3-VL, which builds multilingual capability across all training stages, maintains proactive refusal across languages, whereas shallow multilingual adaptation only during instruction tuning may produce false safety in low-resource languages.