Exploiting the Over-Invariance Caused by Norm-Bounded Adversarial Robustness
Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness
Translation in progress — untranslated text is omitted.
The paper proposes a class of invariance adversarial examples, in which the true label of the input has changed but the model prediction remains unchanged, and argues that perturbation-robust classifiers trained under the ℓp norm are instead more susceptible to this type of attack. The authors use an analytical construction to show that norm-bounded robustness is insufficient to guarantee overall robustness, and conduct experiments on Adversarial Spheres and MNIST to verify this. Human evaluation shows that the ℓ0 and ℓ∞ invariance adversarial examples it generates successfully change the label in 55% and 21% of cases respectively, while these ℓp-robust models agree with human labels on the successful examples at rates below 58% and 5% respectively. The paper also gives an example stating that an ℓ0-robust classifier relying on the sparse Fourier transform achieves only 38% accuracy, lower than the baseline classifier's 54%. The authors accordingly call for more precise classification definitions for adversarial examples and for cautious setting of the radius of the ℓp ball.
- Most adversarial example research targets perturbation-based examples (small input changes, changed model output). The authors argue that pursuing only l_p-norm perturbation robustness may introduce excessive invariance, making models more vulnerable to invariance-based adversarial examples (semantics changed, prediction unchanged).
- The authors give formal definitions of the two types of adversarial examples, constructively analyze the relationship between the two attack types on the synthetic Adversarial Spheres task, and propose a model-agnostic attack: after affine-transforming out-of-class training samples, find the nearest neighbor, then perturb the source image toward it, generating l_0 and l_inf invariance samples on MNIST, with success judged by a 40-person human study.
- In the human study (100 test images), the l_0 attack success rate is 55% and l_inf is 21%. On successful l_0 samples, the l_0 perturbation-robust l_0 Sparse model agrees with human labels 38% of the time, below Baseline's 54%; on successful l_inf samples, five of six models have 0% agreement (Table 1).
- The authors state the attack is very slow (a few minutes per image) and was not optimized; the l_inf attack is weaker, and the authors believe further improvements may be useful; invariance-based robustness still lacks a quantitative metric; and it remains unclear whether the two types of samples, perturbation and invariance, cover all evasion attacks.
Using analytical constructions and human evaluation, the paper shows that robust classifiers trained according to the ℓp norm may instead be more susceptible to invariance adversarial example attacks, adding another dimension to robustness evaluation.