Bridging Mode Connectivity in the Loss Landscape and Adversarial Robustness

1. 01What problem does this paper attempt to solve?

   Use mode connectivity to repair models tampered with by backdoor/error injection, and analyze the robustness loss landscape under evasion attacks.

   Core problem: when a user receives a potentially tampered pretrained model and has only a small amount of clean data, how to eliminate backdoors or injected faults while preserving model accuracy; and whether conventional models and adversarially robust models are connected in the loss landscape.

   • Problem setting: training DNNs is costly, so users often directly use public pretrained models and then fine-tune them, but these models may have been tampered with by attackers; taking backdoor attacks as an example, a tampered model behaves normally when there is no trigger and is difficult to detect (Section 1).
   • Shortcomings of existing methods: the authors state that most backdoor defense research focuses on detecting backdoors or retraining after filtering anomalous training samples, rather than directly repairing the model (Section 2.2); mode connectivity research mainly focuses on generalization analysis and model ensembling, and its implications for adversarial robustness are "largely unexplored" (Section 2.1).
   • Core observation: mode connectivity can find a high-accuracy, low-loss path between two independently trained models, and there are many similarly performing models on the path; the authors hypothesize that there exists a model on this path that both preserves accuracy on clean data and is unaffected by tampering.
   • Proposed in the paper: train a path connecting two tampered models with a small amount of bonafide data, and take a model on the path as the repaired model; and use the same framework to analyze the standard loss and robustness loss landscapes under evasion attacks, and the correlation between robustness loss and the largest eigenvalue of the input Hessian.
   • Threat model:
     • Attacker goal: implant a backdoor during the training phase (10% training data poisoning, single-target or all-target labels) to cause triggered samples to be misclassified; or inject faults into the model weights during the inference phase (fault sneaking attack) to cause 4 target samples to be misclassified.
     • Attacker knowledge (adaptive setting): knows that the defense uses path connection and can use the same path training loss as the defender, but cannot compromise the user's private bonafide data (Section 3.2, Appendix J).
     • Victim system: two (or one) potentially tampered pretrained models held by the user, and the user also has a small amount of bonafide data.
     • Evasion attacks are an inference-phase threat: PGD generates adversarial examples, ε=8/255, 10 iterations.

   Collapse this answer

2. 02What related research exists?

   Related work falls into three groups: mode connectivity; three types of adversarial attacks and their defenses; and research on the geometry of adversarial robustness and "no free lunch."

   Mode connectivity and loss landscape

   • Garipov et al. (2018), Gotmare et al. (2018), Draxler et al. (2018)—proposed and developed mode connectivity: two independently trained models of the same architecture can be connected by simple curves (polygonal chain, Bezier curve) into a high-accuracy low-loss path; Garipov et al. (2018) also used it to design a fast model ensemble method. The path training objective (Equations 1, 2) and curve parameterization (Equations 3, 4) of this paper directly follow this line of work.
   • Yu et al. (2018)—explains adversarial robustness from the perspective of the input-space decision boundary. The paper points out that this work "scratched the surface", considering only evasion attacks, whereas this paper covers training-stage attacks and proposes a repair method.

   Backdoor, evasion and error injection attacks and defenses

   • Backdoor attacks: Gu et al. (2019) (BadNets, the basis for the implementation of the backdoor attack in this paper), Liu et al. (2018); on the defense side, Chen et al. (2018), Wang et al. (2019), Tran et al. (2018) mainly perform backdoor detection or filter anomalous samples and retrain, and the paper points out that the goal of this paper is to directly repair backdoored models.
   • Evasion attacks: Goodfellow et al. (2015), Carlini & Wagner (2017), Madry et al. (2018) (PGD, the basis for the implementation of the evasion attack and adversarial training in this paper); black-box and transfer attacks Papernot et al. (2017), Chen et al. (2017), Zhao et al. (2020).
   • Error injection attacks: Liu et al. (2017), Zhao et al. (2019b) (fault sneaking attack, the basis for the implementation of error injection in this paper), which at the hardware level can be realized by laser beam (Barenghi et al., 2012) and row hammer (Van Der Veen et al., 2016).

   Geometry and theory of adversarial robustness

   • Fawzi et al. (2017; 2018), Wang et al. (2018b), Yu et al. (2018)—study DNN adversarial robustness from a geometric perspective.
   • Tsipras et al. (2019), Dohmatob (2018), Bubeck et al. (2019)—the "no free lunch" hypothesis: adversarially robust models cannot be obtained without additional cost; this paper states that the robustness loss barrier provides geometric evidence for this hypothesis.
   • Moosavi-Dezfooli et al. (2019)—empirically verified that the input gradient is highly aligned with the maximum eigenvector of the input Hessian, supporting the assumption in Proposition 1 of this paper that c is usually large.

   Baselines and benchmarks: the comparison baselines are fine-tune, train from scratch, prune (filter pruning of Li et al., 2017) and random Gaussian perturbation (noisy model); the datasets are CIFAR-10 and SVHN (CIFAR-100 additionally in Appendix N), and the architectures are VGG16 and 26-layer Preactivation-ResNet.

   Authors' positioning: to their knowledge this is "the first work" to propose using the mode connectivity method to study adversarial robustness (Section 1, as stated by the authors).

   Collapse this answer

3. 03How does the paper address this problem?

   Train a Bezier curve path connecting two tampered models with a small amount of bonafide data, and take a model on the path as the repaired model; the same framework is used to analyze the robustness loss landscape.

   Overall idea: treat mode connectivity as a "model repair" tool—although the two models tampered with in the same way are poisoned at the endpoints, the mid-path model of a connection path trained with clean data can both maintain clean accuracy and escape the tampering effect; the same framework is also used to characterize the robustness loss landscape under evasion attacks. The method follows the mode connectivity framework of Garipov et al. (2018).

   Formalization of path connection

   • Let w1, w2 be the weights of two independently trained models, and ϕ_θ(t) (t∈[0,1]) be a parameter curve whose two ends equal w1 and w2 respectively. The training objective is L(θ) = E_{t∼ U(0,1)}[l(ϕ_θ(t))], i.e., the expected loss of models uniformly sampled on the curve, where l is the cross-entropy loss.
   • The curve uses a quadratic Bezier curve: ϕ_θ(t) = (1−t)² w₁ + 2t(1−t)θ + t² w₂, where θ is the only trainable parameter (Equation 4).

   Problem setting for repairing tampered models

   • Two tampered models: the user holds two models tampered with in the same way (the same poisoned data or the same set of target samples) and a small amount of bonafide data, trains the path with the bonafide data, and selects the model at some t on the path as the repaired model.
   • Extension with only one tampered model: first fine-tune the model with bonafide data (2000 images, 100 epochs), then connect the original model with the fine-tuned model (Appendix G).
   • Strategy for choosing t (Appendix Q): the user cannot observe the attack error rate, so it is recommended to choose a model whose test accuracy is not lower than the endpoint model's accuracy minus a threshold ∆a; when the full test set is accessible, ∆a=6% is recommended, and when only bonafide data is available, estimate with 5-fold cross-validation and ∆a=10% is recommended.

   Attack implementation

   • Backdoor attack: implemented following Gu et al. (2019) (BadNets), randomly selecting 10% of training images, adding a white square trigger at the bottom-right corner and changing the labels; the single-target attack target label is fixed to class 1, and the all-target attack target label is the original label plus 1 mod 9 (Section 3.2, Appendix C).
   • Error injection attack: adopts the fault sneaking attack (Zhao et al., 2019b), selecting 1000 images from the test set, of which 4 are target samples, and modifying the model weights so that these 4 are misclassified to the specified target label while the other 996 keep their classification unchanged (Appendix C).
   • Evasion attack: for the 9 models at t={0.1,…,0.9} on the path, PGD (l∞ norm, ε=8/255, 10 iterations) is used to generate non-target adversarial examples over the entire test set; the robustness loss is defined as the cross-entropy between the adversarial example prediction and the original label (Section 3.4).

   Robustness loss landscape and correlation analysis

   • The path is trained with the entire training set using the standard cross-entropy loss, comparing three types of model pairs: standard-standard, standard-adversarial training, and adversarial training-adversarial training (adversarial training uses PGD, ε=8/255, 10 iterations).
   • The maximum eigenvalue λmax(t) of the input Hessian (second derivative of the loss with respect to the input) of each model on the path is computed using power iteration, converging to a relative error of 1E-4 (Appendix K).
   • Proposition 1: under the assumptions that the standard loss is constant on the path, the loss can be approximated by a second-order Taylor expansion, and the normalized inner product of the input gradient with the maximum eigenvector is c, the oracle robustness loss max l(w(t), x+δ) is proportional to λmax(t) as c→1; the proof is in Appendix L.

   Technical explanations and adaptive attack

   • Weight space exploration: add Gaussian noise to the tampered model to generate 1000 noise models, testing whether a good model could be obtained randomly (Appendix H).
   • Input gradient similarity: compare the cosine similarity of input gradients between models on the path and the endpoint models on clean data and tampered data, explaining why the path can neutralize the backdoor effect (Appendix I).
   • Adaptive attack: the attacker trains a "bad path" that is entirely tampered with poisoned data and releases its endpoint models, and the defender then connects these two models with bonafide data (Appendix J).

   Collapse this answer

4. 04What experiments did the paper conduct?

   Evaluated on CIFAR-10/SVHN and VGG/ResNet the repair effect of path connection on backdoor and error injection, and the loss landscape under evasion, including adaptive attacks and multiple ablation groups.

   Experimental setup

   • Evaluated models: VGG16 (13 convolutional layers + 3 fully connected layers) and 26-layer Preactivation-ResNet; the clean test accuracy of the untampered models is shown in Table A1 (e.g., 88%/86% for VGG on CIFAR-10, 97%/99% for ResNet on SVHN).
   • Datasets: CIFAR-10, SVHN; CIFAR-100 (VGG) added in Appendix N.
   • Attack settings: the backdoor attack poisons 10% of the training data (single-target/all-target); the error injection attack injects errors into 4 target samples; evasion uses PGD (ε=8/255, 10 iterations).
   • Baselines: fine-tune, train from scratch, prune (VGG prunes about 60% of parameters, ResNet about 20%), noisy model (add zero-mean Gaussian noise to the two endpoint models, repeat 50 times and average).
   • Metrics: clean accuracy (standard test accuracy), backdoor/injection accuracy (i.e., attack success rate), robustness loss, maximum eigenvalue of the input Hessian and PCC.
   • Sample size and training: both path connection and baselines are trained for 100 epochs; the bonafide data amounts are 2500/1000/500/250/50; the evaluation of backdoor and error injection uses 5000 non-overlapping clean/triggered test images (Table 1 note); the stability experiment is 10 repetitions (Appendix P).

   Main results

   Repair effect under single-target backdoor attack (according to Table 2, path connection takes t=0.1 (CIFAR-10)/t=0.2 (SVHN)):

   Swipe horizontally to read the table

   Dataset (architecture)	Metric	Method	2500	250	50
   CIFAR-10 (VGG)	clean acc	path connection	88%	77%	63%
   CIFAR-10 (VGG)	clean acc	fine-tune	84%	74%	46%
   CIFAR-10 (VGG)	backdoor acc	path connection	1.1%	3.3%	2.5%
   CIFAR-10 (VGG)	backdoor acc	fine-tune	1.5%	1.9%	2.8%
   CIFAR-10 (VGG)	backdoor acc	prune	43%	79%	82%
   SVHN (ResNet)	clean acc	path connection	96%	89%	82%
   SVHN (ResNet)	backdoor acc	path connection	2.5%	4.3%	16%
   SVHN (ResNet)	backdoor acc	fine-tune	14%	63%	60%

   • The authors state that path connection consistently maintains clean accuracy across architectures and datasets and reduces the attack success rate to near 0%; Figure 2 shows that most models with t∈[0.1,0.9] are resilient to the backdoor attack.
   • fine-tune is the best baseline: on CIFAR-10 its clean accuracy is lower than path connection (17 percentage points lower with 50 data); on SVHN its clean accuracy can match but its backdoor accuracy is higher (58.7 percentage points higher with 250 data).
   • The noisy model baseline is ineffective (backdoor accuracy 91%–97%), and prune has high clean accuracy but can hardly reduce backdoor accuracy (Table 2).

   Removal of error injection attack

   • What was tested: connecting two models injected with 4 errors under the same setting. Results: path connection has injection accuracy of 0% under all data amounts, and the highest clean accuracy (92% with 2500 data on CIFAR-10 VGG, 88% with 50, Table 3).
   • Among the baselines, only train from scratch also reaches 0% attack success rate, but its clean accuracy is low (10% with 50 data); fine-tune and prune retain 25% attack success rate under some settings (Table 3).
   • The authors state that this demonstrates a new application of mode connectivity for finding precise and adversarially robust models.

   Robustness loss landscape under evasion attack

   • What was tested: on CIFAR-10 (VGG), paths are trained with standard loss, connecting three pairs of models: standard-standard, standard-adversarial training, and adversarial training-adversarial training, and the standard loss, robustness loss, PGD attack success rate and λmax(t) are measured (Figure 4).
   • Results: the standard loss has no barrier in all three cases; the robustness loss shows a barrier (hill) between standard and adversarially trained models, and the mid-path models of the dual adversarially trained model path are also less robust than both ends. The PCC of λmax(t) with the robustness loss is 0.71, 0.88, 0.86 respectively (Figure 4 caption).
   • Authors' interpretation: the barrier provides geometric evidence for the "no free lunch" hypothesis; on the path trained with standard loss there are "essentially no better adversarially robust models". Under the robust connection (adversarial training path) in Appendix M, a barrier still exists between standard-robust model pairs, with PCC 0.98/0.98/0.94 (Figure A14).

   Adaptive attack

   • What was tested: the attacker trains an entire "bad path" with poisoned data and releases the endpoint models, and the defender then connects them with clean data (Appendix J, CIFAR-10 VGG, single-target backdoor).
   • Results: the attack error rate on the released path is below 5% (10000 test samples, Figure A12(a)); after the defender reconnects, models with t∈[0.25,0.75] can be repaired, and path connection (t=0.27) achieves 90% clean accuracy and 3.8% backdoor accuracy with 2500 data, outperforming the baselines (Table A4). The authors state that the adaptive attack reduces the proportion of robust models on the path but does not break the defense. The path-aware version of error injection can likewise be removed (Figure A13).

   Other ablations and analyses

   • Single-model extension: when there is only one tampered model, fine-tune first and then connect; with 250 data the error rate on triggered samples reaches 100% at t=0.25 (Appendix G, Figure A7).
   • Connecting two backdoor models with different poisoning ratios (30% and 10%) is still effective (Appendix G, Figure A9).
   • Weight space exploration: 1000 Gaussian noise models have low clean accuracy and high attack success rate, and the authors state that good models are unlikely to be obtained randomly (Appendix H, Figure A10).
   • Input gradient similarity: the gradient similarity of mid-path models on triggered/injected data is lower than on clean data, and the authors state that this shows path connection neutralizes the backdoor effect (Appendix I, Figure A11).
   • Model ensemble: ensembling randomly selected models on the path reduces the PGD attack success rate from 85% to 79%, and the authors state that the improvement is not large, possibly due to the transferability of adversarial examples (Appendix M).
   • Stability: 10 repetitions under the CIFAR-10 (ResNet) backdoor setting, with 2500 data at t=0.2, clean accuracy 87±0.21%, backdoor accuracy 1.2±0.47% (Appendix P, Table A5).
   • CIFAR-100 (VGG): the conclusions on backdoor and error injection repair are consistent; under evasion the PCC of standard-standard and standard-adversarial training connections is 0.74/0.98 (Appendix N, Figure A16).

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper has no dedicated limitations section; the experiments cover three attacks, two architectures and three datasets, and adaptive attacks and stability each have dedicated validation.

   Limitations and future directions noted by the authors

   • The paper does not specifically discuss limitations (no Limitations section). Two places explicitly written by the authors that relate to coverage: Appendix N states that the effect of adversarial training on CIFAR-100 is "not as significant as that on CIFAR-10", so the connection of the two adversarially trained models on CIFAR-100 was not studied; Appendix Q points out that the bonafide data of 50 images has a large bias under 5-fold cross-validation, and recommends using a larger amount of bonafide data.

   Experimental coverage

   • The evaluated models are VGG16 and 26-layer Preactivation-ResNet, 2 architectures in total; the datasets are CIFAR-10 and SVHN, with CIFAR-100 (VGG only) added in Appendix N (Section 3, Appendices A, N).
   • The backdoor attack is a BadNets-style trigger (10% poisoning, single-target/all-target); the error injection attack injects only 4 target samples; the evasion attack is only PGD (ε=8/255, 10 iterations) (Section 3.2–3.4).
   • The comparison baselines are fine-tune, train from scratch, prune, and noisy model, 4 in total; the adaptive attack evaluation covers both backdoor and error injection, with the setting that the attacker knows the defense and can use the same path training loss (Section 3.2, Appendix J).
   • Number of repetitions: the noisy model baseline is repeated 50 times and averaged; the stability of path connection is repeated 10 times and the standard deviation reported only under one setting, CIFAR-10 (ResNet) backdoor, and Appendix P states that doing stability analysis for all settings is costly, so only one representative setting was run.
   • The paper does not report the training time or computational cost of path connection (Appendix A only states that 6 GTX 1080Ti GPUs were used).

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors state that mode connectivity can repair tampered models with a small amount of clean data and reveal a robustness loss barrier between conventional and robust models.

   An ICLR 2020 paper that turns mode connectivity from a generalization analysis tool into an evaluation and repair means for adversarial robustness; the authors state that this is the first work to use mode connectivity for adversarial robustness.

   • Problem to solve: publicly released pretrained models may be implanted with backdoors or injected with errors and are hard to detect, and the user has only a small amount of bonafide data, so a repair tool is needed that both preserves clean accuracy and eliminates the adversarial effect; at the same time, the geometric structure of adversarial robustness on the loss landscape is not yet clear.
   • Method highlights: connect two models tampered with in the same way with a quadratic Bezier curve, minimizing only with a small amount of bonafide data (50–2500) the cross-entropy loss of models uniformly sampled on the path, and take the mid-path model as the repaired model; the same framework is used to analyze the standard/robustness loss landscape under evasion attacks.
   • Backdoor repair: under the CIFAR-10 (VGG) single-target backdoor, with 50 bonafide data path connection maintains 63% clean accuracy and backdoor accuracy is only 2.5%, whereas the prune baseline reaches 82% backdoor accuracy (Table 2); on SVHN (ResNet) with 250 data fine-tune's backdoor accuracy is 58.7 percentage points higher than path connection.
   • Error injection removal: path connection reduces the attack success rate of the 4 injected errors to 0% under all data amounts, and its clean accuracy is higher than the other effective baselines (Table 3).
   • Robustness loss landscape: on the path between standard and adversarially trained models the standard loss is flat but the robustness loss has a barrier, which the authors call geometric evidence for the "no free lunch" hypothesis; the PCC of the robustness loss with the maximum eigenvalue of the input Hessian reaches 0.71–0.88 (Figure 4), and a theoretical explanation is given in Proposition 1.
   • Authors' conclusion: path connection remains resilient to adaptive attacks that know the defense mechanism (Appendix J), and the conclusions are consistent across two architectures and multiple datasets; the authors consider mode connectivity a "holistic tool" for evaluating and improving adversarial robustness. Model ensemble yields limited gains against evasion attacks (attack success rate only drops from 85% to 79%, Appendix M).

   Collapse this answer

Related papers