ReACT-CLIP: Response-Aware Test-Time Defense for Vision-Language Models

ReACT-CLIP is a training-free test-time defense for CLIP-like vision-language models that adaptively determines the correction strength and whether to intervene based on the input. It sets the correction strength based on the relative increase in visual feature drift under low- and high-noise probing, and judges the necessity of intervention based on prediction instability under weak spatial augmentation. On 12 downstream datasets and ImageNet and its distribution-shift variants, the method significantly improves robustness under multiple attack types and strengths while largely maintaining clean accuracy.