VLMShield: An Efficient and Robust Defense for Vision-Language Models Against Malicious Prompts
VLMShield: Efficient and Robust Defense of Vision-Language Models against Malicious Prompts
AI summary
Researchers have developed a plug-and-play lightweight safety detector, VLMShield, for identifying multimodal malicious prompt attacks against vision-language models. The method first uses the MAFE framework to let CLIP process long text and fuse multimodal information, then detects attacks using the difference in feature distributions between benign and malicious prompts, addressing the vulnerability caused by visual integration weakening alignment. Experiments show that it performs excellently in robustness, efficiency, and practicality, and the code has been released.