Improving the Transferability of Adversarial Examples with Input Diversity

1. 01What problem does this paper attempt to solve?

   To address the problem that adversarial examples generated by iterative adversarial attacks transfer poorly to black-box models.

   Core question: how to generate adversarial examples with high success rates under both white-box and black-box settings.

   • Problem setting: CNNs perform well on tasks such as image classification, but are vulnerable to additive perturbations imperceptible to humans (adversarial examples); the authors believe that studying the generation of adversarial examples helps examine the shortcomings of model robustness and training algorithms.
   • Shortcomings of existing methods: according to the paper, iterative attacks (such as I-FGSM) have high white-box success rates but easily overfit to specific network parameters and transfer poorly in the black-box setting; single-step attacks (such as FGSM) underfit, have low white-box success rates, and transfer slightly better. There is a trade-off between the two.
   • Core observation: data augmentation can prevent training overfitting; prior work has shown that image transformations can defend against adversarial examples, indicating that adversarial examples generalize poorly under different transformations. Based on this, the authors hypothesize that transformed adversarial examples are "hard examples" for the attacker and can be used to generate more transferable examples.
   • Method proposed in the paper: at each step of the iterative attack, apply a random differentiable transformation (random scaling, random padding) to the input with probability p, proposing DI2-FGSM and its momentum-combined variant M-DI2-FGSM, combined with multi-model ensemble attacks.
   • Threat model:
     • Goal: the generated adversarial examples change the predicted label on both white-box and black-box models.
     • Knowledge: under the black-box setting, the attacker has no knowledge of the model structure and parameters; under the white-box setting, the attacker fully knows the network structure and weights.
     • Capability: iteratively generate perturbations on a local model, with the perturbation constrained by the l∞ norm (ε=15 in the experiments, which the authors state is still imperceptible to humans).
     • Victim systems: ImageNet classification networks (normally trained and adversarially trained) and the defense solutions of the NIPS 2017 adversarial competition.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: adversarial example generation methods, defense methods, and EOT, which is closest to this paper.

   Adversarial example generation

   • Szegedy et al. [36] pointed out that CNNs are vulnerable to adversarial examples and proposed box-constrained L-BFGS; Goodfellow et al. [11] proposed the single-step FGSM; Kurakin et al. [16, 17] extended it to the iterative version I-FGSM and demonstrated adversarial examples in the physical world.
   • Dong et al. [9] proposed the momentum-based iterative attack MI-FGSM to improve transferability; Liu et al. [21] improved transferability by simultaneously attacking an ensemble of models. These two are the baselines directly compared and combined in this paper.
   • Adversarial examples also exist in tasks such as object detection, semantic segmentation, speech recognition, and deep reinforcement learning; Nguyen et al. [25] generated fooling images that are unrecognizable to humans but classified with high confidence by CNNs.

   Defense methods

   • Adversarial training [11, 17] and ensemble adversarial training [37]: injecting adversarial examples into the training data; Tramèr et al. [37] pointed out that such models remain vulnerable to attack.
   • Random image transformations at inference time [38, 12], random activation pruning [8], pixel deflection plus wavelet denoising [27], and purifying adversarial images with generative models [24, 33, 29].

   Relationship to EOT

   • The authors point out that input diversity is related to EOT [2], and list three differences: this paper mainly focuses on the black-box setting while EOT focuses on white-box; this paper aims to mitigate overfitting in attacks, while EOT aims to make adversarial examples robust to transformations and does not discuss overfitting; this paper does not perform an expectation step at each iteration, whereas expectation is the core of EOT.

   Baselines and benchmarks: the compared baselines are FGSM, I-FGSM, MI-FGSM (all implemented in the cleverhans library [26]) and the C&W attack [4]; the evaluation benchmarks are a subset of the ImageNet validation set and the top-3 defenses and official baselines of the NIPS 2017 adversarial competition [18].

   The authors position this paper as: introducing random transformations originally used for defense into the attack process to mitigate overfitting, combinable with momentum and ensemble attacks, and intended as a benchmark baseline for evaluating network robustness and defense effectiveness.

   Collapse this answer

3. 03How does the paper address this problem?

   At each step of the iterative attack, random scaling and padding are applied to the input with probability p, and then the gradient is computed on the transformed input, mitigating overfitting to a single network.

   Overall idea: bring the idea of data augmentation into the attack process—at each iteration of I-FGSM, apply a random differentiable transformation to the current adversarial example with probability p, and maximize the loss on the transformed input, thereby generating adversarial examples that are robust to transformations and more transferable. The method is named DI2-FGSM, and its momentum-combined variant is M-DI2-FGSM.

   Problem formulation

   • Given an image X, the true label y_true, and network parameters θ, the goal is to maximize the loss L(X+r, y_true; θ), subject to the perturbation r satisfying the l∞ norm ||r||∞ ≤ ε and the predicted label y_adv ≠ y_true.
   • The loss function is cross-entropy: L(X, y_true; θ) = −1_ytrue · log(softmax(l(X; θ))), where l(X; θ) is the logits output.

   Motivation: overfitting of iterative attacks

   • The authors analyze: single-step attacks (FGSM) underfit because the linear approximation of the loss is inaccurate, resulting in low white-box success rates; iterative attacks (I-FGSM) greedily perturb along the gradient sign at each step, easily falling into poor local maxima and overfitting to specific parameters θ, so the generated examples rarely transfer to black-box models.
   • Based on this, the authors propose that a better optimization approach is needed to mitigate overfitting; prior work has shown that simple image transformations can invalidate adversarial examples, indicating that transformed adversarial examples can serve as better optimization samples.

   DI2-FGSM and M-DI2-FGSM

   • DI2-FGSM: at each step of I-FGSM, apply the transformation T(·) with probability p, with the update rule X^adv_n+1 = Clip^ϵ_X {X^adv_n + α · sign(∇_X L(T(X^adv_n; p), y_true; θ))}, where T returns the transformed image with probability p and the original image with probability 1−p.
   • Transformation instances: random scaling (scaling the input to a random size) plus random padding (randomly padding zeros around it) [38]; the authors state they also tried rotation and flipping, and random scaling plus padding had the best transferability.
   • Role of p: p controls the trade-off between white-box and black-box success rates; when p=0 it degenerates to I-FGSM, and when p=1 the black-box success rate is higher but the white-box success rate decreases (the attacker does not see the original input).
   • M-DI2-FGSM: replace the gradient of the momentum term with the gradient on the transformed input, i.e., gn+1 = µ·gn + ∇L(T(Xn; p)) / ||∇L(T(Xn; p))||1; the authors state that momentum and input diversity are two different ways of resisting overfitting and can be naturally combined.

   Relationships between attacks and ensemble attacks

   • Through parameter settings they can degenerate into each other (Figure 2): when p=0, M-DI2-FGSM degenerates to MI-FGSM and DI2-FGSM degenerates to I-FGSM; when µ=0, M-DI2-FGSM degenerates to DI2-FGSM; when the number of iterations N=1, I-FGSM degenerates to FGSM.
   • Ensemble attack: following the strategy of [21, 9], fuse the logits of K models: l(X) = Σ wk·lk(X; θk), with wk ≥ 0 and summing to 1; the authors believe that examples that remain adversarial to multiple networks are more likely to transfer to other networks.

   Implementation parameters

   • Step size α=1, total number of iterations N=min(ε+4, 1.25ε), maximum perturbation ε=15; momentum decay factor µ=1; transformation probability p=0.5; the input is first randomly scaled to rnd×rnd×3 (rnd ∈ [299, 330)), then randomly padded to 330×330×3.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Single-model and ensemble attacks were tested on 5000 ImageNet images across 7 networks, and the transferability of the top defenses from the NIPS 2017 competition was reproduced.

   Experimental setup

   • Evaluated networks: 4 normally trained networks Inception-v3 (Inc-v3), Inception-v4 (Inc-v4), Resnet-v2-152 (Res-152), Inception-Resnet-v2 (IncRes-v2); 3 adversarially trained networks Inc-v3ens3, Inc-v3ens4, IncRes-v2ens [37].
   • Dataset: 5000 images randomly selected from the ImageNet validation set that are correctly classified by all evaluated networks, uniformly resized to 299×299×3.
   • Baseline attacks: FGSM, I-FGSM, MI-FGSM (cleverhans implementation); C&W and D-C&W are additionally tested on 1000 images (C&W parameters: maximum iterations 250, learning rate 0.01, confidence 10).
   • Metric: attack success rate (proportion of predicted labels changed); diagonal blocks are white-box, off-diagonal blocks are black-box.
   • Parameters: α=1, N=min(ε+4, 1.25ε), ε=15, µ=1, p=0.5; for ensemble attacks the 6 models are equally weighted wk=1/6.
   • NIPS 2017 competition reproduction: top-3 defenses TsAIL, iyswim, Anil Thomas and 3 official baselines Inc-v3adv, IncRes-v2ens, Inc-v3; following the competition procedure, the 5000 images are split into 50 batches, with ε randomly drawn from {4/255, 8/255, 12/255, 16/255} for each batch; the attack is on an ensemble of 8 models (the first 7 equally weighted 1/7.25, Inc-v3adv weight 0.25/7.25), N=10, µ=1, DI2-type methods p=0.4.

   Main results

   Single-model attacks (generated on each of the 4 normally trained networks, tested on all 7 networks, according to Table 1):

   Swipe horizontally to read the table

   Source model	Attack	Inc-v3	Inc-v4	IncRes-v2	Res-152	Inc-v3ens3	IncRes-v2ens
   Inc-v3	I-FGSM	99.9%	14.8%	11.6%	8.9%	3.3%	1.5%
   Inc-v3	M-DI2-FGSM	99.9%	63.9%	59.4%	47.9%	14.3%	7.0%
   Inc-v4	I-FGSM	22.0%	99.9%	13.2%	10.9%	3.2%	1.7%
   Inc-v4	M-DI2-FGSM	72.4%	99.5%	62.2%	52.1%	17.6%	8.8%
   IncRes-v2	I-FGSM	22.2%	17.7%	97.9%	12.6%	4.6%	2.5%
   IncRes-v2	M-DI2-FGSM	71.2%	67.4%	96.1%	57.4%	25.1%	14.9%
   Res-152	I-FGSM	20.8%	17.2%	14.9%	99.1%	5.4%	2.8%
   Res-152	M-DI2-FGSM	78.9%	76.5%	74.8%	99.2%	35.2%	19.0%

   (The Inc-v3ens4 column and the FGSM, DI2-FGSM, MI-FGSM rows are omitted; see Table 1 for complete data.)

   • The authors state that M-DI2-FGSM substantially outperforms the other baseline attacks on all black-box models, while maintaining a high white-box success rate.
   • Looking at input diversity alone: DI2-FGSM achieves a higher success rate than I-FGSM on black-box models (regardless of adversarial training), while maintaining the white-box success rate; for example, when generated on Res-152, DI2-FGSM achieves 53.8% on Inc-v3 and 11.1% on Inc-v3ens4, versus 20.8% and 4.6% for I-FGSM.
   • The authors state that attacks on black-box adversarially trained networks remain relatively weak: in single-model attacks the highest black-box success rate on IncRes-v2ens is only 19.0%.

   Ensemble attacks (Table 3)

   • Setup: generated on an ensemble of 6 models, tested on the ensemble network (white-box) and the hold-out network (black-box).
   • Results: under the black-box setting M-DI2-FGSM is highest on all hold-out networks, e.g., 80.7% when holding out Inc-v3 (I-FGSM 43.7%, DI2-FGSM 69.9%, MI-FGSM 71.4%), and 44.6% when holding out Inc-v3ens3 (I-FGSM 12.9%, DI2-FGSM 36.3%, MI-FGSM 22.8%).
   • Exception: under the white-box setting DI2-FGSM and M-DI2-FGSM are slightly lower than I-FGSM and MI-FGSM (e.g., 90.1% vs. 96.6% on the ensemble when holding out Inc-v3); the authors explain this as attacking multiple networks simultaneously being harder, and state it can be improved by decreasing p, increasing N, or decreasing α.

   C&W and the NIPS 2017 competition

   • D-C&W (Table 2): on 1000 images, D-C&W achieves a higher success rate than C&W on black-box models, e.g., when generated on Inc-v3 and transferred to Inc-v4 it rises from 5.7% to 16.8%; the white-box success rate remains 100.0%. The authors state that input diversity can generalize to attacks beyond the FGSM family.
   • NIPS 2017 (Table 4): M-DI2-FGSM achieves an average success rate of 73.0%, higher than I-FGSM 62.4%, DI2-FGSM 68.1%, MI-FGSM 66.4%, and the competition's officially reported MI-FGSM* 64.2%; the authors state this exceeds the competition's top-1 attack submission by 6.6%. By defense, M-DI2-FGSM achieves 20.0% on TsAIL, 69.8% on iyswim, and 64.4% on Anil Thomas.
   • Implementation check: the reproduced MI-FGSM result of 66.4% differs from the official MI-FGSM* of 64.2%, which the authors attribute to the randomness of the maximum perturbation magnitude during the attack.

   Other ablations and analyses

   • Transformation probability p (Figure 4): as p increases, the black-box success rate rises and the white-box success rate falls; at smaller p, the black-box success rate rises while the white-box rate drops only slightly (ensemble attack setting).
   • Number of iterations N (Figure 5): from N=15 to 31, both the white-box and black-box success rates of DI2-FGSM rise; M-DI2-FGSM shows a similar trend, but its black-box success rate on adversarially trained models does not improve with more iterations, and the gap between the two methods narrows as N increases.
   • Step size α (Figure 6): p=0.5, N=ε/α; a smaller α improves the white-box success rate; in the black-box setting DI2-FGSM is insensitive to α, while M-DI2-FGSM still improves as α decreases.
   • Visualization (Figure 3): shows 5 randomly selected pairs of clean images and adversarial examples (DI2-FGSM on Inc-v3, ε=15); the authors state the perturbations are imperceptible to the human eye.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors' hypothesis on the cause of transferability remains to be verified; the experiments cover 7 ImageNet classification networks and the NIPS 2017 competition defenses.

   Limitations and future directions noted by the authors

   • The explanation of the method's cause is only a hypothesis (Sec. 4.6 Discussion): The authors propose a hypothesis—different networks, because they are trained on the same dataset (e.g., ImageNet), have decision boundaries with similar intrinsic structure; input diversity makes adversarial examples more robust to small transformations and effective within a region near the decision boundary, thereby more easily fooling other networks; the authors explicitly state they plan to verify this hypothesis theoretically or experimentally in the future.
   • Attacks on black-box adversarially trained networks remain relatively weak (Sec. 4.3): The authors write that in single-model attacks, the black-box success rate on adversarially trained networks remains relatively low (at most 19.0% on IncRes-v2ens), so ensemble attacks are used to further improve transferability.
   • The paper has no dedicated Limitations section; the above two points are the shortcomings and future directions explicitly written by the authors in the Discussion and experimental analysis.

   Scope of experimental coverage

   • The evaluated models are 7 ImageNet classification networks: 4 normally trained (Inc-v3, Inc-v4, Res-152, IncRes-v2) and 3 adversarially trained (Inc-v3ens3, Inc-v3ens4, IncRes-v2ens); the NIPS 2017 experiments additionally include Inc-v3adv (Sec. 4.1, 4.5).
   • The data are 5000 images from the ImageNet validation set that are correctly classified by all evaluated networks; the C&W experiments use 1000 of them (Sec. 4.1, 4.2).
   • The defense evaluation covers the top-3 defenses from the NIPS 2017 competition (TsAIL, iyswim, Anil Thomas) and 3 official baselines; the authors note that due to resource constraints only the top-3 defenses were considered (Sec. 4.5).
   • Attack parameters: ε=15 (NIPS experiments use a random ε per batch), p=0.5 (NIPS experiments p=0.4), µ=1, α=1, N=min(ε+4, 1.25ε) (Sec. 4.1, 4.5).
   • The paper does not report the number of repetitions or variance for each experiment; the ablation of ensemble attacks is conducted only under the ensemble setting (Sec. 4.4).

   Collapse this answer

6. 06Summarize the main content of the paper

   Introduce random input transformations into the iterative attack to mitigate overfitting and improve the black-box transferability of adversarial examples.

   An attack paper on the transferability of adversarial examples: addressing the problem that iterative attacks have high success rates in the white-box setting but transfer poorly in the black-box setting, the authors propose introducing random image transformations into the attack process.

   • Method: DI2-FGSM applies random scaling and random padding to the input with probability p at each iteration of I-FGSM, and computes the gradient on the transformed input; combined with momentum, it becomes M-DI2-FGSM, which can also attack an ensemble of multiple models simultaneously. The authors state that random scaling plus padding transfers better than rotation and flipping.
   • Single-model attack (Table 1): on 5000 ImageNet images and 7 networks, M-DI2-FGSM has a higher black-box success rate than FGSM, I-FGSM, and MI-FGSM, for example 63.9% when generated on Inc-v3 and transferred to Inc-v4 (I-FGSM 14.8%); the white-box success rate remains 99.5%–99.9%.
   • Ensemble attack (Table 3): under the leave-one-out black-box setting, M-DI2-FGSM is highest on all networks, for example 80.7% for leave-one-out Inc-v3 and 44.6% for leave-one-out Inc-v3ens3; on the white-box ensemble it is slightly lower than I-FGSM/MI-FGSM.
   • NIPS 2017 competition (Table 4): M-DI2-FGSM has an average success rate of 73.0% against the top-3 defenses and the official baseline, which the authors state exceeds the competition top-1 attack submission by 6.6%.
   • Remaining issues: the authors state that attacks on black-box adversarially trained networks remain weak (at most 19.0% on IncRes-v2ens for single-model attacks); the explanation of why the method works is only a hypothesis, and the authors plan to verify it in the future. The authors hope this attack can serve as a benchmark baseline for evaluating network robustness and defense effectiveness.

   Collapse this answer

Related papers