Toward Evaluating the Robustness of Neural Networks
Towards Evaluating the Robustness of Neural Networks
Translation in progress — untranslated text is omitted.
Nicholas Carlini and David Wagner propose new adversarial example attacks against three distance metrics, L0, L2, and L∞, successfully finding adversarial examples with 100% probability on both defensive distillation and non-distilled neural networks. Defensive distillation previously reduced the success rate of existing attacks from 95% to 0.5%, but could not stop these stronger attacks. On MNIST, CIFAR-10, and ImageNet, the adversarial examples generated by the new attacks have lower distortion than previous best results, with L0 and L2 attacks having 2 to 10 times lower distortion; the authors state that their L0 attack is the first published attack capable of targeted misclassification on ImageNet. On Inception v3, flipping only the least significant bit of each pixel can change the image classification; the authors also used the transferability of high-confidence adversarial examples to test and break defensive distillation, and released the attack code at http://nicholas.carlini.com/code/nn_robust_attacks，建议作为评估未来防御的基准。
- Defensive distillation was proposed to improve neural networks' robustness to adversarial examples, and the original paper reported that it reduced the success rate of existing attacks from 95% to 0.5%. The authors aim to test whether distillation really eliminates adversarial examples or only makes existing attacks fail.
- The authors formulate adversarial example generation as an optimization problem with box constraints, systematically compare 7 objective functions and 3 box-constraint handling methods, select a logits-based objective function with a tanh variable substitution, then design attacks for L2, L0, and L∞ respectively, and use high-confidence adversarial examples for transfer tests.
- On MNIST, CIFAR-10, and ImageNet, the three attacks succeed 100% of the time on both undistilled and distilled networks; the perturbation of the L0/L2 attacks is 2×–10× lower than that of previous attacks (per the authors); the correlation coefficient between distillation temperature and average perturbation distance is −0.05; high-confidence adversarial examples can transfer from ordinary networks to distilled networks.
- The authors do not have a dedicated Limitations section; the authors point out that none of the Lp distances used is a perfect measure of human-perceived similarity, and leave constructing a better distance metric for future work. The experiments cover only three image tasks, MNIST, CIFAR-10, and ImageNet, and one defense, defensive distillation.
The authors used three new attacks to break defensive distillation with a 100% success rate, showing that evaluating new defenses for image models requires stronger attacks as a benchmark.
Related events and news
0 events · 2 news mentions
News that mentions this paper
- Nicholas Carlini Writingcites the paper's id or linkNicholas Carlini 回顾《Towards Evaluating the Robustness of Neural Networks》五年后的反思
to me that it's been five years since I wrote “Towards Evaluating the Robustness of Neural Networks” (with my at-the-time advisor) and they asked if I had any thoughts on this paper.
- Nicholas Carlini Writingcites the paper's id or linkNicholas Carlini 给出评估对抗样本防御的建议
Among other papers, see Towards Evaluating the Robustness of Neural Networks, Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods, and Obfuscated Gradients Give a False Sense
Related papers
Show 3 more papers
- GeoDetect: Geometric Adversarial Detection for VLPsDefensesarXiv