RobustBench: A Standardized Adversarial Robustness Benchmark

1. 01What problem does this paper attempt to solve?

   Adversarial robustness evaluation is error-prone and often overestimates robustness; the authors propose the standardized benchmark RobustBench to accurately compare defenses within limited compute.

   Core problem: adversarial robustness research lacks systematic, trustworthy standardized evaluation, making it hard to tell which defenses genuinely improve robustness.

   • Setting and importance: the authors state that the field already has over 3000 papers, but which defenses are truly effective and which merely make commonly used attacks overestimate robustness is often unclear; exactly computing robust accuracy is NP-hard even for a single-layer network.
   • Shortcomings of existing methods: the authors state that many seemingly robust defenses have been broken by adaptive attacks; Tramèr et al. observed that even defenses with adaptive evaluation can still be broken by new adaptive attacks; certified methods give lower bounds that often clearly underestimate robustness, so the benchmark measures only upper bounds.
   • Core observation: the authors see recurring patterns across many defenses (zero gradients, randomization, optimization loops at inference time) that defeat standard attacks without necessarily improving robustness, and therefore advocate imposing restrictions on models admitted to the leaderboard.
   • What the paper proposes: the RobustBench benchmark, including a standardized leaderboard based on AutoAttack (120+ models), a Model Zoo with a unified interface (80+ models), and analyses of distribution shift, calibration, OOD detection, fairness, privacy leakage, smoothness, and transferability based on the collected models.
   • Threat models:
     • Attacker knowledge: fully white-box setting, the model is fully known to the attacker.
     • Perturbation sets: l∞ (ε∞=8/255 on CIFAR-10/100, ε∞=4/255 on ImageNet) and l2 (ε2=0.5); thresholds are taken from the literature to ensure the true label is unchanged within the perturbation.
     • common corruptions: no attacker; classification accuracy averaged over corruption types and severities.
     • Victim system: image classifiers satisfying three restrictions (nonzero gradients, deterministic forward pass, no optimization loop at inference time).

   Collapse this answer

2. 02What related research exists?

   Related work covers attack libraries and competitions, existing robustness benchmarks, adaptive attack recommendations, and commonly used test models; the authors state that RobustBench differs from them in standardized worst-case evaluation plus restrictions.

   Attack libraries and competitions

   • Libraries such as FoolBox, Cleverhans, AdverTorch, AdvBox, ART, SecML, and DeepRobust mainly implement popular adversarial attacks, some with basic defenses, but the authors point out that they do not include the latest SOTA models.
   • The two NeurIPS 2017/2018 adversarial competitions [79, 10] sought the most robust models against specific attacks, but had deadlines and can only reflect the best defenses at that time.

   Existing benchmarks and leaderboards

   • DEEPSEC (Ling et al.) tests combinations of various attacks and defenses; the paper relays Carlini's criticism: it reports averages rather than worst-case, the evaluated threat model is inconsistent with training, and perturbations are too large.
   • RayS (Chen and Gu) is a hard-label black-box attack and forms a leaderboard; the authors point out that its robust accuracy still tends to be overestimated compared with the original evaluations.
   • Dong et al. evaluated a small number of defenses (3 l∞ and 2 l2 on CIFAR-10); the authors point out that it does not include some of the best defenses and attacks, some results are higher than in the original papers, and it imposes no model restrictions.
   • RobustML collects robustness claims and external evaluations of defenses; the authors point out that it relies entirely on community submissions, has no baseline attacks, and submissions are not frequent enough to provide an up-to-date overview.

   Adaptive attacks and standard test models

   • Athalye et al., Carlini et al., Tramèr et al., among others, proposed recommendations for adaptive attacks and showed that several defenses can be broken; Tramèr et al. observed that many defenses that had undergone adaptive evaluation were still broken by new attacks.
   • New attacks are often tested on the public models of Madry et al. and Zhang et al.; the authors point out that only two models per dataset do not constitute a sufficiently large testbed, and repeated evaluation may cause attacks to overfit to these two defenses.

   Baselines and evaluation method: RobustBench uses AutoAttack (Croce and Hein) as the standardized evaluation attack, including white-box and black-box attacks; it also welcomes external evaluations such as adaptive attacks as a supplement.

   Authors' positioning: compared with the above benchmarks, RobustBench is characterized by (1) worst-case baseline evaluation using a strong attack ensemble and integration of external evaluations, (2) adding reliability flags to AutoAttack, (3) threat models consistent with training and clearly defined, and (4) evaluating the latest defenses rather than only standard defenses, with an open-source Model Zoo.

   Collapse this answer

3. 03How does the paper address this problem?

   RobustBench forms a continuously updatable robustness benchmark through four parts: model restrictions, standardized evaluation with AutoAttack, reliability flags, and the Model Zoo.

   Overall approach: under explicit threat models and model restrictions, use the parameter-free attack ensemble AutoAttack to give an upper bound on robust accuracy, supplemented by a flag mechanism and external adaptive evaluation for correction, and distribute models uniformly via the Model Zoo. The framework is named RobustBench.

   Problem setting and formalization

   • For a classifier f and perturbation set ∆, a successful adversarial perturbation δ satisfies argmax_c f(x+δ)_c ≠ y and δ∈∆; robust accuracy is defined as the fraction of samples correctly predicted for all perturbations in ∆.
   • Exact computation is infeasible; the benchmark reports only upper bounds obtained from attacks, and the authors state that upper bounds are usually much tighter than certified lower bounds.
   • Threat models: fully white-box; on CIFAR-10/100, ε∞=8/255 and ε2=0.5; on ImageNet, ε∞=4/255; common corruptions are evaluated as average accuracy over corruption types and severities.

   Model restrictions

   • Classifiers admitted to the leaderboard must satisfy: input gradients generally nonzero; fully deterministic forward pass; no optimization loop in the forward pass.
   • The authors' rationale: non-certified defenses that violate these restrictions often merely make gradient attacks harder rather than genuinely improving robustness, and lack a standardizable reliable evaluation; if reliable standardized evaluation methods emerge in the future, the restrictions will be relaxed.

   Evaluation procedure

   • l∞/l2 defenses are evaluated with AutoAttack: four attacks run sequentially—PGD with automatically tuned step size (using cross-entropy loss and DLR loss respectively), targeted FAB, and the black-box Square Attack; subsequent attacks only attack points not yet successfully attacked.
   • Rationale (per the authors): it includes both white-box and black-box attacks, requires no hyperparameter tuning, and gives lower robust accuracy than the original papers report for almost all models.
   • Standardized evaluation is run by the benchmark independently of the model authors to ensure reproducibility; if other attacks give a lower robust accuracy, the leaderboard lists the worst-case result separately.
   • Reliability flag: a warning is output when Square Attack lowers robust accuracy by more than 0.2% more than the white-box gradient attacks in AutoAttack, indicating that an adaptive attack may be needed; an automatic flag for violations of the three restrictions is also integrated.

   New defense submission and Model Zoo

   • New entries must satisfy the three restrictions, have a public paper, and share model checkpoints; entries lacking a checkpoint may be temporarily listed but marked unverified and may be removed.
   • The Model Zoo collects checkpoints of leaderboard models (with the authors' authorization); currently only PyTorch implementations are accepted, all ResNet/WideResNet variants; models can be loaded by identifier in two lines of code.
   • The number of models for each dataset and threat model is shown in Table 1 (e.g., CIFAR-10 l∞: 39 in the Model Zoo, 63 on the leaderboard).

   Success criterion and reproducibility

   • Robustness is determined by the upper bound on robust accuracy given by AutoAttack (or a lower external evaluation); common corruptions are determined by average accuracy.
   • Appendix E reports the variation and runtime of AutoAttack across four random seeds (Table 2), and fixes torch==1.7.1 and torchvision==0.8.2 to ensure reproducibility of ImageNet results.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The experiments include leaderboard evaluation of 120+ models, AutoAttack reproducibility and flag cases, and analyses based on the Model Zoo of distribution shift, calibration, OOD, fairness, privacy, smoothness, and transferability.

   Experimental setup

   • Evaluated models: the leaderboard has 120+ models in total (Table 1: CIFAR-10 l∞ 63, l2 18, corruptions 15; CIFAR-100 l∞ 14, corruptions 4; ImageNet l∞ 6, corruptions 7), with architectures such as ResNet/WideResNet variants; the analysis section mainly uses CIFAR-10 l∞ models.
   • Datasets: CIFAR-10, CIFAR-100, ImageNet (a fixed 5000-image validation subset); distribution shift analysis uses CIFAR-10-C, CIFAR-10.1, CINIC-10; OOD detection uses CIFAR-100, SVHN, Describable Textures.
   • Evaluation method: AutoAttack ensemble (APGD-CE, APGD-DLR, targeted FAB, Square Attack); the fairness experiment uses APGD-DLR (3 target classes, 20 iterations each), which the authors state differs from the full evaluation by less than 0.5% on average.
   • Metrics: clean/robust accuracy, ECE (15 equal-width bins, grid search over temperature t∈[0.001,1.0] with step 0.001), AUROC, RSD, membership inference accuracy, local Lipschitz constant (N=256, PGD 50 steps, step size ε/5).
   • Repetitions: the reproducibility experiment runs 4 random seeds for each of 4 models (Table 2); the transferability experiment uses two attacks, 10-step PGD (step size ε/4) and FGSM.

   Main results

   Leaderboard main results (CIFAR-10, per Table 3, Table 4):

   Swipe horizontally to read the table

   Model	l∞ clean	l∞ robust	l2 clean	l2 robust
   Rebuffi et al. (WRN-70-16, extra)	92.23	66.56	95.74	82.32
   Gowal et al. (WRN-70-16, extra)	91.10	65.87	94.74	80.53
   Rebuffi et al. (WRN-106-16)	88.50	64.58	not reported	not reported
   Rade and Moosavi-Dezfooli (WRN-34-10, extra)	91.47	62.83	not reported	not reported
   Carmon et al. (WRN-28-10, extra)	89.69	59.53	not reported	not reported
   Madry et al. (WRN-34-10)	87.14	44.04	not reported	not reported
   Xiao et al. (DenseNet-121)	79.28	7.15	not reported	not reported
   Standard (WRN-28-10)	94.78	0.0	94.78	0.00

   • The authors state that AutoAttack gives lower robust accuracy than the original papers for almost all models (middle panel of Figure 2), and that reported values for several published defenses were overestimated.
   • The authors observe that the top of the l∞ leaderboard still consists of variants of PGD adversarial training, improved with extra data, early stopping, weight averaging, etc.; extra data alleviates the robustness-accuracy trade-off, but the clean accuracy of highly robust models remains below that of standard-trained models.
   • The best l2 model has higher standard accuracy than the standard model (95.74% vs. 94.78%), and robust accuracy 82.32% vs. 0.00% (Appendix F).

   Flag mechanism and adaptive evaluation case

   • What was tested: in the Xiao et al. model, the black-box Square Attack lowered accuracy by more than 10 percentage points more than the white-box attacks, triggering the flag.
   • Result: with an adaptive Square Attack using 30 random restarts (originally 1), robust accuracy drops from AutoAttack's 18.50% to 7.40%; the original paper reports 52.4%. In Table 3 this model is listed as 7.15%.
   • Authors' interpretation: AutoAttack did not completely fail (it already revealed robustness below the reported value), but the exact value was overestimated; currently only this model is flagged on the leaderboard.

   Reproducibility and runtime

   • What was tested: AutoAttack repeated with 4 random seeds for each of 4 models across different threat models/datasets (Table 2).
   • Results: Gowal et al. (CIFAR-10 l∞) 62.82% ± 0.016, 11.8 h; Rebuffi et al. (CIFAR-10 l2) 78.80% ± 0.000, 15.1 h; Wu et al. (CIFAR-100 l∞) 28.84% ± 0.018, 6.6 h; Salman et al. (ImageNet l∞) 25.31% ± 0.010, 1.6 h (single Tesla V100).
   • Authors' interpretation: results across seeds are very close; models with lower robustness are evaluated faster (attacks stop once an adversarial example is found).

   Property analyses based on the Model Zoo

   • Distribution shift: on CIFAR-10, gains in robust accuracy correlate with gains in standard accuracy on CIFAR-10-C/10.1/CINIC-10 (Figure 3); l2 robust models improve markedly on CIFAR-10-C; robust accuracy across datasets is also positively correlated (Figure 10).
   • Calibration: most l∞ robust models are underconfident (optimal temperature below 1); uncalibrated ECE averages 21%, up to 70% for the Pang et al. model, versus 4% for standard models; temperature scaling narrows but does not fully eliminate the gap (Figure 4). Two l2 models are better calibrated: Engstrom et al. 1.41% vs. 3.71% for the standard model (unscaled), Gowal et al. 1.00% vs. 1.11% (after scaling) (Figure 11).
   • OOD detection: l∞ robust training reduces AUROC relative to standard training; extra data helps under standard training but gives no similar improvement under robust training; l2 models are broadly comparable to standard models, with Augustin et al. better (Figure 5, 12).
   • Fairness: robust accuracy varies widely across classes, with a marked drop for the "deer" class under l∞; higher average robust accuracy corresponds to lower RSD, but MART has a clearly higher RSD at around 57% robustness (Figure 6).
   • Privacy leakage: membership inference accuracy varies widely across robust models, some lower than standard models, with no strong correlation with robust accuracy and mainly determined by the generalization gap; l2 models are higher than l∞ models (Figure 7, 14, 15).

   Other ablations and analyses

   • Smoothness: standard models have the highest local Lipschitz constant at each layer, and robust models are smoother; the output-layer Lipschitz constant shows a clear relationship with robust accuracy (Figure 16, 17).
   • Transferability: adversarial examples transfer strongly between robust→robust, non-robust→robust, and non-robust→non-robust, and transfer weakly from robust→non-robust, under both l∞ and l2 (Figure 18, 19).
   • ImageNet preprocessing: different PyTorch/torchvision versions can change standard accuracy by up to 0.16% (5000 points), so versions are fixed (Appendix E).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list limitations such as potential bias introduced by the restrictions and insufficient representativeness of threat models, and plan to extend to new threat models and tasks; the experiments are currently limited to three image classification datasets.

   Limitations and future directions noted by the authors

   • The restrictions may introduce community bias and suppress research in certain directions; the authors state that without these restrictions reliable evaluation would not be possible, and they are working to relax them as much as possible (Appendix A).
   • Good performance on the benchmark does not guarantee safety in real deployment; l_p and corruption robustness may not be sufficient to represent all real-world threat models (Sec. 5, Appendix A).
   • common corruptions are artificially generated and may differ from real-world corruptions (Appendix A).
   • Future plans: keep the leaderboard updated and add new datasets and new threat models, with candidates including sparse perturbations (l0/l1, adversarial patches), multi-l_p-norm perturbations, adversarially optimized common corruptions, and perturbation sets unseen during training (Sec. 5).
   • Another direction is developing standardized evaluation for defenses based on test-time adaptation (which do not satisfy the "no optimization loop" restriction) (Sec. 5).
   • The authors believe the benchmark's structure and principles can generalize to tasks and domains such as image segmentation, image retrieval, natural language processing, and malware detection, and welcome contributions from each community (Sec. 5).

   Scope of experiments

   • The evaluated models are the 120+ image classification models on the leaderboard, with architectures such as ResNet/WideResNet variants; the Model Zoo includes 80+ of them (Table 1).
   • The datasets are CIFAR-10, CIFAR-100, and ImageNet (a fixed 5000-image validation subset); the threat models are l∞, l2, and common corruptions.
   • Property analyses (calibration, OOD, fairness, privacy, smoothness, transferability) are conducted mainly on CIFAR-10 l∞ models; some results for l2 and CIFAR-100 are in Appendix F.
   • Reproducibility experiments cover 4 models × 4 random seeds (Table 2); the fairness experiment uses APGD-DLR with a reduced budget (3 target classes × 20 iterations).
   • The paper does not report the total number of queries for evaluating each leaderboard model; the transferability experiments use only two attacks, PGD-10 and FGSM, to generate adversarial examples.

   Collapse this answer

6. 06Summarize the main content of the paper

   RobustBench is an adversarial robustness benchmark with model restrictions and standardized AutoAttack evaluation, comprising a leaderboard of 120+ models, a Model Zoo of 80+ models, and multi-dimensional property analyses.

   RobustBench is a standardized benchmark for adversarial robustness in image classification, aimed at addressing the problem that robustness evaluation is error-prone and defense progress is hard to compare.

   • Problem addressed: The authors state that many defenses merely make commonly used attacks overestimate robustness; exact evaluation is infeasible and certified lower bounds are too loose, so reliable upper-bound evaluation and explicit threat models are needed.
   • Method highlights: Models admitted to the leaderboard must have nonzero gradients, deterministic forward passes, and no optimization loop at inference time; evaluation uses AutoAttack (a sequential ensemble of APGD-CE, APGD-DLR, FAB, and Square Attack) in a fully white-box setting for l∞ (8/255, ImageNet 4/255) and l2 (0.5), with a flag indicating possible overestimation when black-box attacks are clearly better than white-box ones; the Model Zoo uniformly provides 80+ models.
   • Main results: The leaderboard includes 120+ models; the top CIFAR-10 l∞ entry, Rebuffi et al., has robust accuracy 66.56%, and the top l2 entry has 82.32% (Table 3, 4); the authors state that AutoAttack is lower than the original paper's reported value for almost all models; the flagged Xiao et al. model drops from 18.50% to 7.40% with 30 restarts of Square Attack; across four random seeds, AutoAttack varies by no more than ±0.018 (Table 2).
   • Analysis findings: The authors state that most l∞ robust models are underconfident and worse calibrated (average uncalibrated ECE 21%, standard models 4%); robust training reduces OOD detection; privacy leakage has no strong correlation with robustness and is mainly determined by the generalization gap; robust models are smoother; adversarial examples transfer weakly from robust to non-robust models.
   • Authors' conclusion: A standardized benchmark plus flags and adaptive supplementary evaluation can give as accurate a robustness assessment as possible under limited resources; they also caution that good benchmark performance does not mean safe real deployment, and plan to extend to new threat models, new tasks, and standardized evaluation of test-time defenses.

   Collapse this answer

Related papers