Ensemble Adversarial Training: Attacks and Defenses
Ensemble Adversarial Training: Attacks and Defenses
Translation in progress — untranslated text is omitted.
The paper points out that adversarial training with single-step attacks such as FGSM and Step-LL converges to a degenerate minimum: what the model learns is a loss surface that invalidates the linear approximation of gradients, not genuine resistance to strong perturbations. The authors verify this gradient masking effect on Inception v3, Inception ResNet v2 and MNIST models on ImageNet; for example, the approximation ratio of Step-LL to loss maximization drops from 19% for standard Inception v3 to 7% for v3adv after adversarial training. The authors accordingly propose R+FGSM: first add a random step, then perform single-step linearization. At ε=16/256 and α=ε/2, its top-1 error rate on v3adv is 64.8%, higher than Step-LL's 26.6% and the two-step iterative attack's 58.3%. On defense, the authors propose ensemble adversarial training: augmenting training data with perturbations transferred from other pretrained models.
- Can single-step adversarial training at ImageNet scale really withstand black-box attacks? The authors claim that this type of training converges to a degenerate global minimum: the model produces sharp curvature near data points, making single-step attacks weaker rather than genuinely robust.
- The authors formalize and empirically demonstrate the gradient masking phenomenon of single-step adversarial training, and propose the R+FGSM attack, which first adds a small random step and then linearizes, as well as Ensemble Adversarial Training, which augments the training data with transfer perturbations generated by other pretrained models.
- On v3adv, the Step-LL approximation ratio drops from 19% to 7%; R+Step-LL white-box error rate is 64.8%, far higher than Step-LL's 26.6%; IRv2adv-ens has a worst-case Top 1 error rate of 27.0% against transfer attacks on held-out models, and placed first in the first round of the NIPS 2017 defense competition.
- The authors state that non-transfer types of black-box attacks (e.g., generative, interactive query attacks) were not considered; subsequent work shows that stronger transfer attacks can reduce IRv2adv-ens accuracy to 22%, and multi-step adversarial training has surpassed this method.
The paper uses approximation ratios and transfer attack experiments to show that single-step adversarial training merely masks gradients, and presents ensemble adversarial training as a transferable defense approach.
Related events and news
0 events · 1 news mentions
News that mentions this paper
- Nicholas Carlini Writingcites the paper's id or linkNicholas Carlini 给出评估对抗样本防御的建议
For example, a weak version of adversarial training See Figure