"The Boy Who Cried Wolf": Multimodal Guardrails Adversarially Misjudge Safe Inputs as Unsafe

The paper proposes Unsafe Induction Attacks, which add imperceptible perturbations to safe images to induce multimodal guardrail models to judge normal user requests as unsafe and refuse them, creating a "crying wolf" effect that reduces service availability and erodes user trust. The authors propose Unsafe Semantic Distillation (USD), which aligns adversarial perturbations with the distributional representations of unsafe content rather than targeting a specific prompt. Across four advanced guardrail models and simulated scenarios close to real users, USD reports an 84% attack success rate, higher than existing methods; the paper states that threats in this false-positive direction had not been studied before.