Proper network interpretability helps adversarial robustness in classification

1. 01What problem does this paper attempt to solve?

   The authors study how the explanation discrepancy measure affects the difficulty of ISA, and propose a regularization defense based solely on robust explanations.

   Core question: Can adversarial examples fool the classifier while also evading network explanation checks? What exactly is the relationship between explanation robustness and classification robustness, and can "robust explanations" directly buy "robust classification."

   • Problem setting: CNNs are vulnerable to adversarial example attacks; Xu et al. (2019b;a) claim that adversarial perturbations cause differences in explanation maps that can be used to distinguish adversarial examples; whereas Zhang et al. (2018) and Subramanya et al. (2018) claim that interpretation smuggling attacks (ISA, misclassification while keeping explanation maps similar) are not harder. The authors argue that existing work has no consistent conclusion on the relationship between the two.
   • The authors' core observation: Figure 1 shows that prediction-evasion attacks change the explanation maps of both the true class y and the target class y', so measuring explanation discrepancy by a single class alone may be "inappropriate," and ISA is not necessarily truly easy.
   • The paper proposes: an l1-norm 2-class explanation discrepancy measure and its lower-bound theory (Proposition 1), as well as interpretability-aware robust training methods Int and Int2 that penalize only explanation discrepancy and do not use an adversarial loss.
   • Threat model:
     • Attacker goals: three types—misclassification (PGD, unforeseen attacks); keeping the prediction unchanged but maximizing explanation discrepancy (AAI); misclassification while minimizing explanation discrepancy (ISA).
     • Capability: add l∞-norm-bounded perturbations to the input image; robust training uses ε=0.3 (MNIST), 8/255 (CIFAR-10), 0.003 (R-ImageNet).
     • Victim system: CNN image classifiers with a "global average pooling → softmax" structure (Small, Pool, WResnet), with CAM as the interpreter (the theory applies to interpreters satisfying the completeness axiom).

   Collapse this answer

2. 02What related research exists?

   Related work falls into four categories: adversarial attacks and defenses, network interpretation methods, research on the relationship between explanations and robustness, and recent robust attribution training.

   Adversarial attacks and defenses

   • Attack side: Goodfellow et al. (2015), Carlini & Wagner (2017), etc. construct imperceptible perturbations; Eykholt et al. (2018), Brown et al. (2017), etc. study adversarial patches in the physical world.
   • Defense side: defensive distillation (Papernot et al., 2016b), adversarial training (Madry et al., 2018), TRADES (Zhang et al., 2019), input gradient/curvature regularization (Ross & Doshi-Velez, 2018; Moosavi-Dezfooli et al., 2019). The authors state that this paper differs from these works in that it understands and improves robustness from the perspective of explanations.

   Network interpretation methods

   • CAM-type: CAM (Zhou et al., 2016), GradCAM (Selvaraju et al., 2017), GradCAM++ (Chattopadhay et al., 2018), which give class-discriminative localization maps.
   • PSM-type: vanilla gradient (Simonyan et al., 2013), guided backpropagation (Springenberg et al., 2014), SmoothGrad (Smilkov et al., 2017), IG (Sundararajan et al., 2017); IG satisfies the completeness axiom but is computationally intensive.

   Relationship between explanations and adversarial robustness

   • Xu et al. (2019b;a): adversarial perturbations cause changes in class-specific explanation maps, and the authors accordingly consider explanation discrepancy usable as a detection indicator.
   • Zhang et al. (2018), Subramanya et al. (2018): propose ISA, claiming that under certain conditions making an attack invisible to the interpreter is not harder than generating ordinary adversarial examples; Section 3 of this paper re-examines this conclusion and argues that its single-class discrepancy measure is inappropriate.
   • Ghorbani et al. (2019), Dombrowski et al. (2019): claim that explanations themselves are fragile to perturbations; this paper uses their AAI attack to evaluate explanation robustness.

   Most related work and baselines

   • Chen et al. (2019): robust attribution regularized training based on IG. The authors state that it is more computationally heavy and degrades under strong attacks, whereas this paper's method is lighter and applies to any interpreter satisfying the completeness axiom.
   • Comparison baselines and benchmarks: baselines are Normal, Adv (Madry et al., 2018), TRADES, IG-Norm, IG-Norm-Sum, Int-one-class; benchmarks are MNIST, CIFAR-10, Restricted ImageNet.

   Collapse this answer

3. 03How does the paper address this problem?

   First propose an l1 2-class explanation discrepancy measure with a theoretical lower bound, then perform robust training using worst-case explanation discrepancy as a regularizer.

   Overall idea: use an appropriate explanation discrepancy measure to link "explanation robustness" with "classification robustness," then use that measure as a training regularizer to obtain interpretability-aware robust training (Int / Int2).

   Explanation discrepancy measure and theoretical lower bound

   • Define a general discrepancy D(x,x′)=(1/|C|)∑_{i∈ C}|I(x,i)−I(x′,i)|_p; taking C={y, y'} and p=1 gives the l1 2-class discrepancy D2,l1; 2 classes are chosen because Figure 1 shows that the attack changes the explanation maps of both y and y'.
   • Proposition 1: If the interpreter satisfies the completeness axiom, then D_2,ℓ1(x,x′) ≥ 1/2(f_y(x)−f_y′(x)), i.e., the explanation discrepancy of a successful attack has a perturbation-independent lower bound; pushing the discrepancy below the lower bound makes misclassification infeasible. Appendix A gives the proof and generalization, and Appendix E reports that this bound is empirically fairly tight.

   Using ISA to validate the choice of measure

   • Formulate ISA as an optimization problem: a C&W-type attack loss (reaching −τ on success, with τ=0.1 in the paper) plus λ times D2,l1, subject to ‖δ‖∞≤ε; bisect on λ until the attack just fails to succeed, solve with PGD, and consider only targeted attacks.
   • Use this framework to compare l1/l2 × 1/2/all-class measures and test under which measure ISA has the hardest time suppressing explanation discrepancy (results in question 4).

   Discrepancy measure without target labels

   • The defender does not know y' in advance, so softmax confidence is used to weight all non-true classes to approximate the target-class discrepancy, giving D̃ (Equation 7); when the attack succeeds, the top-1 class is the target class, and its discrepancy is penalized the most.

   Int and Int2 training

   • The training objective is the standard loss plus γ times the worst-case explanation discrepancy, with γ=0.01 (chosen based on the sensitivity experiments in Appendix F).
   • Int: D̃worst takes the maximum of D̃ under perturbation (the attacker seeks misinterpretation).
   • Int2: first generate misclassified samples with adversarial perturbations, then compute their D̃ (the attacker seeks misclassification); training uses alternating optimization: the inner loop generates x', and the outer loop fixes x' and minimizes the regularized loss.
   • Variants Int-Adv and Int2-Adv replace the standard loss with an adversarial loss; the interpreter is CAM, because it is computationally light and the experimental models all have a "global average pooling → softmax" structure.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Classification and explanation robustness are evaluated on three datasets under three types of attacks: PGD, unforeseen, and AAI; under large perturbations, the ATA of Int/Int2 is higher than that of Adv.

   Experimental setup

   • Models evaluated: Small (three-layer CNN, 16/32/100 filters, for MNIST and CIFAR-10), Pool (two-layer CNN, for MNIST, results in Appendix H), WResnet (Wide ResNet ×1, for CIFAR-10 and R-ImageNet).
   • Datasets: MNIST, CIFAR-10, Restricted ImageNet (R-ImageNet, taken from Tsipras et al., 2019).
   • Baselines: Normal, Adv, TRADES, IG-Norm, IG-Norm-Sum, Int-one-class; variants Int, Int-Adv, Int2, Int2-Adv. IG-Norm and IG-Norm-Sum are excluded on WResnet due to the computational cost of IG.
   • Metrics: adversarial test accuracy (ATA, 200-step l∞-PGD, 200 random test points); explanation robustness uses Kendall's Tau rank correlation (averaged over 200 random test points, higher is more robust).
   • Training perturbation: ε=0.3 (MNIST), 8/255 (CIFAR-10), 0.003 (R-ImageNet); γ=0.01.
   • AAI setting: l∞ norm, 200 steps, step size 0.01, top-K evaluation with K=8 (Appendices C, D).

   Main results

   ATA under 200-step PGD (according to Table 1):

   Swipe horizontally to read the table

   Method	MNIST Small ε=0.3	MNIST ε=0.4	CIFAR-10 WResnet ε=8/255	CIFAR-10 ε=10/255	R-ImageNet ε=8/255
   Normal	0.000	0.000	0.060	0.060	0.040
   Adv	0.890	0.000	0.170	0.085	0.070
   TRADES	0.885	0.000	0.170	0.100	not reported
   IG-Norm	0.005	0.000	not reported	not reported	not reported
   Int-one-class	0.125	0.080	0.065	0.025	not reported
   Int	0.790	0.140	0.270	0.210	0.280
   Int2	0.845	0.385	0.290	0.220	0.265
   Int2-Adv	0.840	0.620	0.335	0.285	not reported

   • The authors state: when ε reaches or exceeds the training value, the ATA of Int and Int2 is higher than that of Adv (0.270 and 0.290 vs. 0.170 on CIFAR-10 with ε=8/255), and the advantage grows as the attack strengthens; this relies only on explanation regularization, without an adversarial loss.
   • The authors state: IG-Norm fails on MNIST with ε=0.3 (ATA 0.005), and Int-one-class is far worse than Int, supporting the importance of the 2-class measure.
   • The authors state: ATA decreases as ε increases and as the number of PGD steps increases (Appendix G, Table A3), which does not match the two typical behaviors of obfuscated gradients.

   Unforeseen attacks

   • What is measured: the Gabor, Snow, and JPEG l∞/l2/l1 attacks of Kang et al. (2019), with parameters following that paper, on CIFAR-10 Small, 200 random test points (Table 2).
   • Results: Int has the highest ATA in the whole table on JPEG l1 at 0.295; on JPEG l∞, IG-Norm-Sum is highest at 0.480 (Int 0.440); on Gabor, TRADES is highest at 0.220 (Int 0.160); on Snow, Int2-Adv is highest at 0.135.
   • The authors state: Int and Int2 outperform IG-Norm under Snow and JPEG lp attacks, and are comparable to methods that use an adversarial loss.

   Explanation robustness (AAI)

   • What is measured: Kendall rank correlation between original and adversarial CAM under AAI attacks (Table 3).
   • Results: on MNIST with ε=0.3, Int2-Adv 0.931, Int 0.913, Adv 0.857, Normal −0.085; on R-ImageNet with ε=8/255, Int2 0.952, Int 0.946, Adv 0.884.
   • The authors state: Normal does not automatically have explanation robustness; the explanation robustness of Adv and TRADES becomes worse than that of explanation-regularization methods as ε increases.

   Other ablations and analyses

   • ISA measure comparison (Appendix B, Table A1): with the interpretation method fixed, the 2-class measure has the largest NDS and the smallest NSL; with l1+2-class fixed, IG is the hardest to bypass with ISA, but generating ISA with IG on WResnet+R-ImageNet is infeasible within 200 hours.
   • Training cost (Figure 3): training time per epoch for IG-type methods is ≥4 times that of Int; methods using an adversarial loss have an ATA of about 80% at ε=0.3.
   • γ sensitivity (Appendix F, Figure A1): on MNIST, ATA no longer increases after γ=0.01; on CIFAR-10 with εtrain=6/255, going from γ=0.005 to 0.01 raises ATA by 13.0% and lowers clean accuracy by 5.5%.
   • Feature visualization (Figure 4): the authors state that Int and Int2, like Adv, learn texture-aligned robust features, while Normal and IG-Norm do not; the figure shows only visualization plots of each method under different seeds, with no numerical annotations.
   • Exception: on CIFAR-10 Small, Int-Adv's ATA drops to 0.000 under 200-step PGD (Table A4); the paper gives no explanation.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors propose only one direction of semi-supervised extension; experiments are limited to three image datasets, three architectures, and the CAM explainer.

   Limitations and future directions noted by the authors

   • Conclusion: the authors plan to extend the method to semi-supervised settings, incorporating unlabeled training data.
   • The paper does not specifically discuss other limitations.

   Scope of experimental coverage

   • The models evaluated are 3 architectures in total: Small, Pool, and WResnet; the datasets are MNIST, CIFAR-10, and R-ImageNet (Section 5).
   • Defense evaluation covers 200-step PGD (multiple ε, multiple step counts), the 5 unforeseen attacks of Kang et al. (2019), AAI, and four types of ISA attacks (Tables 1–3, Appendix B).
   • The interpreter in the main experiments is CAM; the ISA measure analysis additionally covers GradCAM++, IG, and the penultimate-layer representation Repr (Appendix B).
   • IG-type methods are excluded on WResnet due to computational cost; generating ISA with IG on WResnet+R-ImageNet is infeasible within 200 hours (Section 5, Appendix B).
   • PGD and AAI evaluations each use 200 random test points; the paper does not report multiple repetitions or variance (Section 5, Appendix C).

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors use the l1 2-class explanation discrepancy to link explanation robustness with classification robustness, and propose the Int/Int2 defenses without an adversarial loss.

   A defense paper: addressing the controversy over "whether adversarial examples can fool both the classifier and the interpreter at the same time," the authors propose an l1-norm 2-class explanation discrepancy measure and use it to build interpretability-aware robust training.

   • Problem: Existing work reaches opposite conclusions on whether ISA is easy; the authors argue the reason lies in how explanation discrepancy is measured, and that single-class measures create the illusion that "ISA is easy."
   • Method: Proposition 1 gives a lower bound for D2,l1 of ½(fy(x)−fy'(x)); on the defense side, softmax weighting approximates the target-class discrepancy, and Int and Int2 are trained with a regularizer that minimizes the worst-case explanation discrepancy (based on CAM, γ=0.01), without using an adversarial loss.
   • Result 1 (Table 1): On MNIST Small with 200-step PGD, ε=0.4, Int2 ATA is 0.385 and Int2-Adv 0.620, while Adv and TRADES are 0.000; on CIFAR-10 WResnet with ε=8/255, Int is 0.270 and Int2 0.290, higher than Adv's 0.170.
   • Result 2 (Table 3): Explanation robustness is higher under AAI; on R-ImageNet with ε=8/255, Int2's Kendall correlation is 0.952 and Adv's is 0.884.
   • Result 3 (Figure 3): Training time for IG-type methods is ≥4 times that of Int; IG-Norm achieves an ATA of only 0.005 under PGD on MNIST with ε=0.3.
   • Authors' conclusion: When the explanation discrepancy measure is chosen appropriately, adversarial examples can hardly hide from the interpreter; improving explanation robustness alone can yield classification robustness, and under large-perturbation attacks it outperforms the compared adversarial training methods; the authors plan to extend to semi-supervised settings in future work.

   Collapse this answer

Related papers