ViT-EnsembleAttack: Enhancing Ensemble Models to Improve the Adversarial Transferability of Vision Transformers

The researchers propose ViT-EnsembleAttack, which is claimed to be the first ensemble-based adversarial attack method designed specifically for Vision Transformer. The method adversarially enhances substitute ViTs using three strategies—multi-head dropout, attention score scaling, and MLP feature mixing—with parameters optimized via Bayesian optimization, together with automatic reweighting and step size enlargement modules. Experiments show it significantly improves the adversarial transferability of ensemble attacks on ViTs, greatly surpassing existing methods.