Risk-Aware Injection: Calibrating VLM Safety Without Sacrificing Usefulness

Wang et al. propose Risk-Aware Injection (RAI), a training-free framework that restores safety calibration in VLMs by amplifying unsafe signals in the visual embedding space. The method targets the cross-modal safety gap: LLMs that reliably reject harmful text requests often comply when the same content is presented as an image, because safety alignment from RLHF or instruction tuning primarily acts on text representations, while the vision encoder follows a separate pathway. RAI constructs an unsafe prototype subspace from language embeddings of known unsafe content, identifies visual tokens whose embeddings are close to this subspace at inference time, and modulates their representations to trigger the model's existing safety mechanisms. The authors state that the method substantially reduces attack success rates across multiple multimodal jailbreak benchmarks while maintaining task performance on standard VLM evaluations. They also suggest the approach could extend to embodied AI, such as robot VLA models; whether it can withstand adaptive attacks specifically targeting the unsafe prototype subspace remains an open question.