STRAP-ViT: Separating Tokens and Applying Random Transformations to Defend Against Adversarial Patches in ViT
STRAP-ViT: Segregated Tokens with Randomized -- Transformations for Defense against Adversarial Patches in ViTs
AI summary
The researchers propose STRAP-ViT to defend against adversarial patch attacks on ViT, with robust accuracy only 2-3% away from the clean baseline, outperforming SOTA. The method uses Jensen-Shannon divergence to detect tokens containing adversarial noise, then applies random composite transformations to them; it is training-free and plug-and-play. It is tested on ViT-base-16 and DinoV2 against four attacks: Adversarial Patch, LAVAN, GDPA and RP2.