On the limitations of the MagNet defense against adversarial examples based on the L1 distortion metric

1. 01What problem does this paper attempt to solve?

   The authors state that the L1-distortion-based EAD attack can bypass MagNet under the oblivious black-box setting, weakening its threat model assumptions.

   Core question: Under the oblivious (black-box) attack setting, can adversarial examples based on the L1 distortion metric bypass MagNet defenses?

   • Problem setting: DNN image classifiers are susceptible to adversarial examples, and adversarial examples are transferable—examples generated on an attacker's own DNN can fool the target DNN without knowledge of the target model's details, so transfer attacks are widely used to evaluate defenses under the black-box setting.
   • State of existing methods: The authors state that MagNet (consisting of a detector and a reformer) is one of the stronger defenses under the oblivious black-box setting, able to defend against L2 distortion-based C&W attacks (including high-confidence samples) as well as FGSM, iterative FGSM, and DeepFool.
   • Core observation: The C&W attack is a pure L2 distortion method, whereas EAD uses both L1 and L2 distortion; the authors argue that the L1 term can filter out perturbations to unimportant pixels, thereby generating adversarial examples with better attack performance.
   • Paper contributions: The authors use EAD to attack MagNet and several of its robust variants on MNIST and CIFAR-10, reporting that under the default setting about 90% (MNIST) and 80% (CIFAR-10) of EAD samples can bypass MagNet, whereas C&W achieves only 10% and 52%; the authors state this weakens the assumption that bypassing MagNet requires knowledge of the defense technique (gray-box setting). The authors state this is the first work to show under the oblivious setting that MagNet is not robust to L1 adversarial examples.
   • Threat model:
     • Attacker goal: Generate untargeted transferable adversarial examples that cause the target DNN deploying MagNet to misclassify.
     • Attacker knowledge: Oblivious (black-box) setting, where the attacker is completely unaware of the existence of the MagNet defense.
     • Attacker capability: Generate adversarial examples on an undefended DNN and transfer them to a DNN of the same architecture protected by MagNet.
     • Victim system: DNN image classifiers on MNIST and CIFAR-10 deployed with MagNet (detector + reformer).

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: adversarial examples and transfer attacks, the C&W attack and EAD, and research on breaking defenses.

   Adversarial examples and transfer attacks

   • Szegedy et al. [1], Goodfellow et al. [2] — point out that DNNs are vulnerable to carefully designed adversarial perturbations, which can be visually imperceptible; Evtimov et al. [3], Athalye and Sutskever [4], Kurakin et al. [5] extend adversarial examples to the physical world.
   • Papernot et al. [6], Chen et al. [7] — construct black-box attacks when model details are completely unknown; Liu et al. [8], Papernot et al. [9] study the transferability of adversarial examples.

   C&W attack and EAD

   • Carlini and Wagner [10] (C&W attack) — generates highly transferable adversarial examples by adjusting the confidence parameter; it is the L2 baseline attack compared in this paper.
   • Chen et al. [17] (EAD) — elastic-net attack, combining L1 and L2 distortion; it is the attack method used in this paper; C&W is a special case of EAD when β=0.

   Research on breaking defenses

   • The paper states that various defenses and adversarial subspace analysis methods have been broken or bypassed by subsequent attacks (possibly requiring increased attack strength) [10]–[15], including Carlini and Wagner bypassing ten detection methods [11], an L1 attack on the Madry defense model [12], an analysis of limitations of the LID method [13], Obfuscated Gradients [14], and bypassing Feature Squeezing [15].
   • Carlini and Wagner [20] — claim to break MagNet, but the paper points out that they use a stronger gray-box threat model: the attacker knows the deployed defense technique (auto-encoder) but not the specific parameters; this paper instead bypasses MagNet under the weaker oblivious setting.

   Baseline and benchmarks: The comparison baseline is the C&W (L2) attack, the benchmarks are MNIST and CIFAR-10, and the defense targets are MagNet [16] and its robust variants.

   The authors position this paper as: under a weaker oblivious threat model than [20], using the L1-distortion EAD to bypass MagNet, thereby weakening the threat model assumptions required to bypass MagNet.

   Collapse this answer

3. 03How does the paper address this problem?

   Use EAD to generate L1 transfer adversarial examples on an undefended DNN, attack MagNet and its robust variants, and measure ASR.

   Overall approach: under the oblivious black-box setting, use EAD, which combines L1 and L2 distortion, to generate untargeted transfer adversarial examples on an undefended DNN, attack the same DNN deployed with MagNet, and compare with the pure-L2 C&W attack.

   The attacked defense: MagNet

   • Two modules: The detector computes the Jensen-Shannon divergence (JSD, with temperature parameter T) between F(x) and F(AE(x)); if it exceeds a threshold, the input is judged adversarial. The reconstruction loss kx − AE(x)k can also be used as a detector. The reformer uses an auto-encoder AE(x) to project the input back onto the training data manifold before classification.
   • Default setting: The auto-encoder is learned by minimizing the mean squared error kx − AE(x)k₂² on the training set; MNIST by default uses only two detectors based on L1 and L2 reconstruction error, while CIFAR-10 by default additionally has two JSD detectors with T=10 and T=40.

   Attack method: EAD

   • Optimization objective: min_x c · f(x, t) + |x − x₀|₂² + β|x − x₀|₁, subject to x ∈ [0,1]ᵖ; when β=0 it reduces to the C&W attack.
   • Attack loss: The untargeted version is f(x) = max{[Logit(x)]_{t0} − max_{j≠t0}[Logit(x)]_j, −κ}, where κ is the confidence, controlling attack transferability (larger κ means a stronger attack and greater distortion).
   • Role of the L1 term: The authors believe that β, through ISTA (iterative shrinkage-thresholding algorithm), zeroes out perturbations to unimportant pixels and shrinks perturbations to important pixels at each iteration, thereby generating sparser adversarial examples with better attack performance.

   Experimental parameters

   • Attack parameters: c is determined by a 9-step binary search (starting from 0.001), with 1000 iterations per attack and an initial learning rate of 0.01; β takes values 10⁻³, 10⁻², 5·10⁻², 10⁻¹, and the final sample selection uses two decision rules, EN rule and L1 rule.
   • Confidence range: [0, 40] for MNIST and [0, 100] for CIFAR-10, both with a step size of 5.
   • Samples: 1000 correctly classified test images randomly selected from each of the two datasets.

   Success criterion

   • ASR: The proportion of adversarial examples that are neither detected by the detector nor correctly classified after processing by the reformer; the defender's metric classification accuracy is complementary to ASR, and a higher ASR means a weaker defense.

   Robust MagNet variants

   • MNIST: D+JSD (adding two JSD detectors with T=10, 40), D+256 (increasing the auto-encoder's number of convolution kernels from 3 to 256, architecture in Table II), D+256+JSD (both simultaneously).
   • CIFAR-10: D+256 (increasing the number of convolution kernels to 256, architecture in Table V).
   • Loss function variant: The auto-encoder is instead trained with mean absolute error (L1 loss), tested separately on the two datasets (Figure 12, 13).

   Collapse this answer

4. 04What experiments did the paper conduct?

   EAD's ASR reaches 90.2% (MNIST) and 79.8% (CIFAR-10) on default MagNet, and still up to 94.3% and 93.7% on robust variants.

   Experimental setup

   • System under test: DNN image classifiers on MNIST and CIFAR-10 (following the architecture and training parameters of the MagNet paper [16]), deployed with the default MagNet and robust variants, respectively.
   • Attack methods: EAD (EN rule and L1 rule, β ∈ {10⁻³, 10⁻², 5·10⁻², 10⁻¹}) and C&W (L2) attacks, all untargeted transfer attacks.
   • Datasets: MNIST (handwritten digits 0–9), CIFAR-10 (10 classes); 1000 correctly classified test images randomly selected from each.
   • Metrics: ASR (attack success rate, %) and the defender's classification accuracy; distortion metrics L1 and L2 are averaged over successful samples only.
   • Reproduction note: The authors state they could not reproduce the defense effectiveness of the original MagNet paper; all results come from their own trained MagNet.
   • Hardware: Intel Xeon E5-2620v4 CPU, 125 GB RAM, NVIDIA TITAN Xp GPU.

   Main results

   ASR (%) of different attacks on default MagNet (according to Table I):

   Swipe horizontally to read the table

   Attack	Condition	MNIST ASR	CIFAR-10 ASR
   C&W (L2)	κ=15 (MNIST) / κ=20 (CIFAR-10)	10 (best)	52
   EAD (EN rule)	β=10⁻³	46.2	69.2
   EAD (EN rule)	β=10⁻²	87.8	74.5
   EAD (EN rule)	β=5·10⁻²	90.1	77
   EAD (EN rule)	β=10⁻¹	90.2	78.6
   EAD (L1 rule)	β=10⁻³	70.2	60.5
   EAD (L1 rule)	β=10⁻²	84.5	66.7
   EAD (L1 rule)	β=5·10⁻²	80.5	75.9
   EAD (L1 rule)	β=10⁻¹	83.8	79.8

   • The authors state that EAD's ASR is higher than C&W's, which they consider evidence for the role of L1 distortion in constructing effective adversarial examples; on MNIST, each EAD row has κ=15, and on CIFAR-10, each EAD row has κ=15.
   • The main text gives another measure: at β=10⁻¹ and confidence 15, MagNet's classification accuracy on MNIST drops from about 90% for C&W to 9.7%, i.e., about 90% of EAD samples bypass MagNet.
   • The authors consider this result to mean that a threat model weaker than the gray-box setting of [20] (completely unaware of the defense's existence) is already sufficient to successfully attack on MNIST.

   Attacks on robust MagNet variants

   • MNIST (Table IV): On D+JSD, EAD's highest ASR is 55.6 (EN rule, β=10⁻¹); on D+256 it is 94.3; on D+256+JSD it is 66.3; the main text states that D+JSD has accuracy above 96% against C&W at all confidence levels, but about 40% of EAD samples can still bypass it; D+256+JSD still cannot defend against about 50% of EAD samples.
   • CIFAR-10 (Table VII): On D+256, EAD's highest ASR is 93.7 (L1 rule, β=10⁻¹), versus a maximum of 79.8 under the default setting; the authors state that as β increases, EAD maintains high ASR under both settings.
   • Test set accuracy cost (Tables III, VI): MNIST without MagNet is 99.42%, dropping to 97.55% with D+256+JSD; CIFAR-10 without MagNet is 86.91%, with default MagNet 83.33%, and with D+256 83.4%.

   Effect of auto-encoder training loss

   • What was tested: Changing the auto-encoder training loss from mean squared error to mean absolute error (L1 loss), to verify whether the vulnerability to L1 attacks stems from L2 reconstruction error.
   • Results: The authors report that MagNet trained with L1 loss can defend against C&W L2 attacks on both MNIST and CIFAR-10, but remains vulnerable to EAD (figures in supplementary materials Figures 12, 13).
   • Authors' interpretation: The vulnerability to L1 adversarial examples is not caused by L2 reconstruction error.

   Other ablations and analyses

   • β and decision rules: When β is small (e.g., 10⁻³), the L1 rule outperforms the EN rule (the authors state that L2 distortion dominates at this point); when β is large, the EN rule is better (the authors attribute this to over-shrinkage in the ISTA step) (Section III-B1).
   • Effect of confidence: The authors state that as confidence increases, the detector rejects more samples; on MNIST default MagNet, a defense trough appears in the confidence range 10–20 (the reformer fails while the detector is not yet effective) (Figure 2 (a)).
   • CIFAR-10 default MagNet: At confidence 10, about 70% of EAD samples are neither detected nor corrected, with the greatest vulnerability in the confidence range 10–20 (Figure 3 (a)).
   • Supplementary materials: Figures 4–11 give full curves for each variant under four schemes: no defense / detector only / reformer only / full MagNet.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors suggest that defenses test both L1 and L2 samples and reinforce defenses at medium confidence; the experiments cover only two image datasets.

   Limitations and future directions noted by the authors

   • Evaluate both types of distortion simultaneously: The authors suggest that future defenses test robustness against both L1 and L2 adversarial examples, because the two have different characteristics (Section IV).
   • Defense gap at medium confidence: The authors state that under the oblivious setting, neither MagNet's reformer nor its detector can effectively defend against adversarial examples at medium confidence levels, requiring additional defense mechanisms (Section IV).
   • Reproduction limitation: The authors state they could not reproduce the defense effectiveness demonstrated in the original MagNet paper; all results in this paper correspond to their own trained MagNet (Section III-A).
   • The paper has no dedicated Limitations section; the above is what is explicitly stated in the Conclusion and Discussion and the experimental setup.

   Scope of experiments

   • System under test: One DNN classifier each on MNIST and CIFAR-10 (following the architecture of [16]), deployed with default MagNet and robust variants (three for MNIST, one for CIFAR-10), plus an auto-encoder variant trained with L1 loss.
   • Attack methods: Only C&W (L2) and EAD, both untargeted transfer attacks; the paper does not report results for targeted attacks.
   • Sample size: 1000 randomly selected, correctly classified test images per dataset; the paper does not report the number of repeated experiments.
   • Threat model: Only the oblivious (black-box) transfer setting; the paper does not conduct experiments under gray-box or white-box settings.
   • Datasets: Only two image classification datasets, MNIST and CIFAR-10.

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors state that EAD with L1 distortion can bypass MagNet and its robust variants under the oblivious black-box setting, weakening its threat model assumptions.

   An attack paper targeting adversarial example defenses: under the oblivious (black-box) transfer attack setting, it examines whether adversarial examples based on L1 distortion can bypass the MagNet defense that previously defended against L2 attacks.

   • Problem: MagNet (detector + auto-encoder reformer) can defend against C&W (L2) attacks under the oblivious setting; prior work that broke MagNet [20] required knowing that the defense used an auto-encoder (gray-box setting).
   • Method: On an undefended DNN, EAD (combining L1 and L2 distortion, with β controlling the strength of the L1 term) is used to generate untargeted transfer adversarial examples, attacking the same DNN deployed with MagNet; 1000 test images each for MNIST and CIFAR-10, compared with the C&W attack.
   • Main results: Under default MagNet, EAD's ASR reaches up to 90.2% on MNIST (EN rule, β=10⁻¹, κ=15) and up to 79.8% on CIFAR-10 (L1 rule, β=10⁻¹, κ=15), whereas C&W reaches only 10% and 52% (Table I).
   • Robust variants: After adding a JSD detector, increasing the auto-encoder's number of convolution kernels to 256, or both, EAD's highest ASR still reaches 94.3% (MNIST, D+256, Table IV) and 93.7% (CIFAR-10, D+256, Table VII); training the auto-encoder with L1 loss is also still bypassed by EAD.
   • Authors' conclusion: MagNet and its robust variants can defend against L2 attacks but are not robust to the L1-distortion EAD; the weaker oblivious threat model is already sufficient to bypass MagNet, weakening the assumption in the literature that bypassing MagNet requires knowledge of the defense technique.
   • Authors' recommendation: Future defenses should test robustness against both L1 and L2 adversarial examples, and fill the defense gap where both the detector and reformer fail at medium confidence levels.

   Collapse this answer

Related papers