Why Does Weak-OOD Work? Further Understanding Jailbreaks of Vision-Language Models

The researchers found that mild out-of-distribution (OOD) perturbations achieve better jailbreak results on vision-language models (VLMs) than clean inputs and overly strong perturbations, and define this non-monotonic pattern as weak-OOD. The paper explains this phenomenon by the trade-off between two dominant factors: input intent perception and model refusal triggering; the evidence given by the authors shows that the two factors respond differently to OOD perturbations, consistent with the gap between broad pretraining robustness and narrower safety alignment. Drawing on OCR capability enhancement during pretraining, the authors design an extended method for embedded-text jailbreaks, JOCR (Jailbreak via OCR-Aware Embedded Text Perturbation), which they claim achieves the best average attack success rate (ASR) among the evaluated baselines. The code is open-sourced at https://github.com/Yuxuan2003/weak-ood-jailbreak。