MOAT: Defending against ViT efficiency degradation attacks with model-agnostic random transformations

MOAT is a model-agnostic preprocessing defense pipeline that combines multiple input transformations to protect ViTs using token pruning from adversarial efficiency attacks. The defense acts directly on the input without modifying the model architecture or the token pruning mechanism. Experimental results show that on all tested ViT models, MOAT limits GFLOPs degradation under attack to within 3.4% of the unattacked model.