Skip to content

Transductive learning for adversarial robustness

Towards Adversarial Robustness via Transductive Learning

arXiv· arXiv:2106.08387· Jiefeng Chen, Yang Guo, Xi Wu, Tianqi Li, Qicheng Lao, Yingyu Liang, Somesh JhaUniversity of Wisconsin-Madison; Google; Mila - Quebec Artificial Intelligence InstituteIndexed by this site · 23 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors formalize a threat model for transductive robustness, propose GMSA in the attack model space that breaks defenses such as RMC, and state that ATRM raises robustness on CIFAR-10 from 41.06% to 53.53%.

Threat modelTransductive threat model (Definition 1): The attacker perturbs the clean test set V at test time to obtain U', and the defender uses the transductive learning algorithm Γ to update the model based on U' and then predict on U'. White-box assumption: the attacker knows all details of Γ, but does not know the defender's private randomness.

Questions
Transductive defenses dynamically retrain the model at test time based on the (possibly perturbed) input; in theory the attack becomes a bilevel optimization, and the authors believe this may increase the difficulty of adaptive attacks, but there is a lack of a systematic threat model and strong adaptive attacks to test this.
Methods
The authors formalize the transductive threat model, propose the principle of attacking the model space and instantiate it as GMSA (PGD over the set of historical models); they also propose the ATRM defense combining adversarial training and unsupervised domain adaptation, and give a separation theorem between transductive and inductive settings.
Experiments and results
GMSA breaks RMC, RMC+ and DANN: for example, on MNIST the robustness of RMC(Standard) drops from 98.30% under PGD to 0.50% under GMSA-AVG. The authors state that ATRM still outperforms adversarial training alone under their strongest attack (53.53% vs. 41.06% on CIFAR-10).
Limitations and future work
The authors note that ATRM is computationally expensive, that the results are preliminary, and that the separation theorem applies only to the constructed problem instances. The experiments cover three datasets, MNIST, CIFAR-10, and GTSRB; the URejectron experiment uses only ResNet18; the paper does not report the number of attack queries or the time cost.
Experimental setupResNet18、Standard et al. · MNIST、CIFAR-10 et al. · Robustness、Accuracy et al.
Evaluated models
ResNet18StandardMadry et al.DANNATRM
Benchmarks
MNISTCIFAR-10GTSRB
Metrics
RobustnessAccuracyerrrej
AI summaryThe researchers propose a model-space attack principle for solving bilevel optimization attack objectives against transductive defenses, and the authors state that the instances they give can break prior transductive defenses. Such defenses dynamically retrain the model at test time based on the input, and attacks against them can be reduced to bilevel optimization. The authors point out that these attacks show clear difficulties in using transductive learning to improve adversarial robustness, while also providing theoretical and experimental evidence supporting the utility of transductive learning.

The researchers propose a model-space attack principle for solving bilevel optimization attack objectives against transductive defenses, and the authors state that the instances they give can break prior transductive defenses. Such defenses dynamically retrain the model at test time based on the input, and attacks against them can be reduced to bilevel optimization. The authors point out that these attacks show clear difficulties in using transductive learning to improve adversarial robustness, while also providing theoretical and experimental evidence supporting the utility of transductive learning.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Establish a threat model for transductive robustness, propose adaptive attacks that break existing defenses, and provide new evidence supporting transductive learning.

    Core question: whether transductive learning can truly improve adversarial robustness—there is a lack of both a unified threat model and strong adaptive attacks that can test such defenses.

    • Problem setting: Classical adversarial robustness is an inductive setting: the model is fixed after training, and the attacker attacks with perturbed inputs. Recent work (Goldwasser et al. [11], Wu et al. [33]) has the defender update the model at test time using a transductive learning algorithm Γ based on (possibly perturbed) unlabeled inputs U', and then predict; the authors believe this has practical motivation in batch prediction deployment scenarios.
    • Shortcomings of existing methods: The authors believe that previous evaluations of attacks against such defenses were weak—the attack used by RMC [33] is not an adaptive attack (the attacker does not know the defense mechanism); techniques such as BPDA [3] targeting test-time defenses are also not applicable, because transductive learning is far more complex than those defenses that do not update the model.
    • Core observation: Attacking a transductive defense is a bilevel optimization; Γ is non-differentiable and the attack set U' appears on both the attack and defense sides. The authors' key observation is to exploit the transferability of adversarial examples to change the target to attacking an entire family of models (model space) induced by the neighborhood of U', thereby relaxing the attacker-defender coupling constraint.
    • Contributions of the paper: Formalize the transductive threat model (Definition 1); propose the principle of attacking the model space and its instance GMSA, which breaks RMC, RMC+, DANN; propose the ATRM defense and prove a separation theorem between transductive and inductive settings (Theorem 1).
    • Threat model (Definition 1, terminology copied as is):
      • Attacker goal: Under the agreed perturbation type, perturb the clean test set V into U', maximizing the loss La of the updated model Γ(F, D, U') on V'.
      • Knowledge: white-box—the attacker knows all details of the defender's transductive mechanism Γ, but does not know Γ's private randomness.
      • Capability: The attacker cannot directly attack the model finally trained by the defender, and can only choose a perturbation set within the neighborhood of U'.
      • Victim system: The defender is a (T, Γ) pair—first train F on D using supervised algorithm T, then at test time use Γ(F, D, U') to produce an updated model and evaluate only on U'.
  2. What related research exists?

    Related work spans three directions: adversarial robustness, transductive learning and domain adaptation; the paper focuses on testing the two transductive defenses [11] and [33].

    Adversarial robustness in the inductive setting

    • Attacks and the principle of adaptive attacks: [13, 4, 19, 24, 9] propose various attacks when the model is fixed; [29] develops the principle of adaptive attacks, and [3] (BPDA) based on these principles breaks many defenses that rely on obfuscated gradients.
    • Adversarial training: [23, 34] are the basic methods for obtaining robustness in the inductive setting; [25, 5] study the amount of data required for robust generalization and the role of unlabeled data.

    Adversarial robustness in the transductive setting

    • Goodfellow [12]: Proposes a research agenda for dynamic models, where the model keeps changing after deployment to defend against associated attacks.
    • Wu et al. [33] (RMC): The authors state that this is the first defense method in this setting, performing runtime masking and cleansing on individual points at test time. The paper points out that the attack used in its evaluation is not an adaptive attack, and this paper breaks it using the method of attacking the model space.
    • Goldwasser et al. [11] (Rejectron/URejectron): Gives transductive guarantees under arbitrary perturbations and bounded VC dimension. The paper empirically studies URejectron in the deep learning small-perturbation setting and finds two failure modes.

    Domain adaptation and test-time adaptation

    • DANN [2, 10]: A classical unsupervised domain adaptation method. The paper proposes a new use of DANN as a transductive defense, and states that when used alone it is broken by model-space attacks, but when combined with adversarial training (ATRM) it yields non-trivial robustness improvements.
    • Test-time adaptation [32, 28]: Test-time adaptation methods for distribution shift. The authors state that these methods are not designed for adversarial robustness, and find that they are fragile even under transfer attacks (the main text does not give corresponding experimental tables).

    Baselines and benchmarks: The attack baselines are PGD [23] (including the PGD-skip setting proposed by [33]) and transfer attacks; the defense baselines are Standard (standard training) and Madry et al. [23] (adversarial training). The datasets are MNIST [20], CIFAR-10 [18], GTSRB [27].

    The authors position this paper as work connecting three directions: testing transductive defenses with the principle of adaptive attacks, while providing new theoretical and empirical evidence for the usefulness of transductive learning.

  3. How does the paper address this problem?

    Formalizes the transductive threat model, constructs the GMSA attack using the attack model space principle, and proposes the ATRM defense and a separation theorem.

    The overall approach has three parts: first, unify transductive defenses into a game-theoretic threat model; then, targeting its bilevel structure, propose the attack model space principle and its instance GMSA; finally, give positive evidence—the ATRM defense and a transductive/inductive separation theorem.

    Threat model formalization

    • Game structure (Definition 1): the defender first trains F = T(D) on a clean training set D; the attacker, after receiving a clean test set V, produces a perturbed set V' (V' ∈ N(V)) and sends the feature part U' to the defender; the defender uses Γ(F, D, U') to produce an updated model F*; the judge computes La(F*, V').
    • Attack objective: maxV′ ∈ N(V),  U′ = V′|X La(Γ(F, D, U′), V′)—U' appears on both the attack and defense sides, which is the key difference from inductive attacks.
    • Scope: the authors say the framework can cover test-time defenses [3], Randomized Smoothing [7], RMC [33], and URejectron [11]; this paper focuses on the setting where Γ genuinely performs transductive learning (updates the model), with small perturbations, deep learning, and no redaction.

    Attack difficulties and the attack model space principle

    • Three difficulties: (D1) Γ is an optimization algorithm, so the objective becomes bilevel optimization, and Γ is non-differentiable; (D2) U' is both the attack target and the defense input, making it hard to find an update direction that defeats the model it itself induces; (D3) the output of Γ can be a random variable due to private randomness.
    • Principle: exploit the transferability of adversarial examples and change the objective to the robust version maxU′ minU ∈ N(U′) La(Γ(U), U′)—find an attack set that defeats the entire model space induced by the neighborhood of U'. The authors believe this relaxes the attacker-defender constraint and provides more information for handling non-differentiability.

    Two attack algorithms

    • FPA (Fixed Point Attack, Algorithm 1): adapted from the literature on solving bilevel optimization in deep learning [22]: in round i, fix model F(i) to construct attack set U(i), then run Γ to obtain F(i+1); after T rounds, select the best attack set. The authors found FPA to be weak in experiments.
    • GMSA (Greedy Model Space Attack, Algorithm 2): in round i, attack all previous historical models {F(0),…,F(i)}, with the loss taken as LAVG (average, giving GMSA-AVG) or LMIN (minimum, giving GMSA-MIN), solved with PGD; likewise iterate T rounds and select the best. In experiments T = 9.

    ATRM defense and separation theorem

    • ATRM (Equation (10)): write the model as F(x) = c(φ(x)), and the training objective is the loss on the adversarially perturbed training set D' plus α times the representation distribution distance d(p_φ(D'), p_φ(U')) between D' and U', i.e., a combination of adversarial training and unsupervised domain adaptation (DANN-style representation matching).
    • Separation theorem (Theorem 1): the authors prove that for any ε ∈ (0, 1/3), there exists a binary classification problem instance such that under the inductive setting any algorithm has loss at least (1−ε)/2, while under the transductive setting there exists a polynomial-time algorithm with loss at most ε. The proof construction is based on a Gaussian data model (Appendix C.2).
  4. What experiments did the paper conduct?

    On MNIST/CIFAR-10, GMSA breaks RMC, RMC+, and DANN; the authors say ATRM still outperforms standalone adversarial training under their strongest attack.

    Experimental setup

    • Defenses/models tested: RMC, RMC+ (extended version), DANN, ATRM, each paired with a Standard (standard training) or Madry et al. (adversarial training [23]) pretrained model; URejectron is evaluated separately on GTSRB with ResNet18.
    • Datasets: MNIST, CIFAR-10 (main experiments); GTSRB (URejectron experiment, taking 10 classes with 10,000 augmented images per class, split 63,000/7,000/30,000 into training/validation/test).
    • Attack configuration: MNIST uses L∞ PGD, ε = 0.3, step size 0.01, 100 steps (200 steps for the DANN/ATRM experiments); CIFAR-10 uses L∞ PGD, ε = 8/255, step size 1/255, 40 steps (100 steps for the DANN/ATRM experiments); FPA/GMSA iterate T = 9.
    • Metrics: Robustness (accuracy under attack) and Accuracy (clean accuracy); unless stated otherwise, values are percentages.
    • Repetition and randomness: the defender uses private randomness different from the attacker's; the PGD-skip experiment is independently repeated 1000 times and averaged; DANN/ATRM are repeated with 5 random seeds (Figure 3).
    • URejectron experiment details: x̃ consists of 50% normal samples and 50% adversarial samples; the PGD attack uses ε = 8/255 and 40 steps; the CW attack uses the L2 norm, c = 1, κ = 0, and 100 steps; corruption uses brightness from [15] with severity 1.

    Main results

    According to Table 3 (DANN and ATRM, PGD as a transfer attack; "—" means not reported in the paper):

    Swipe horizontally to read the table

    DatasetMethodAccuracyPGDFPAGMSA-AVGGMSA-MIN
    MNISTStandard99.420.00———
    MNISTMadry et al.99.1691.61———
    MNISTDANN99.2796.6696.8179.376.17
    MNISTATRM99.0295.5595.1594.3295.22
    CIFAR-10Standard93.950.00———
    CIFAR-10Madry et al.86.0641.06———
    CIFAR-10DANN92.0554.298.550.510.08
    CIFAR-10ATRM85.1160.7161.5953.5357.66
    • The authors say DANN used alone has non-trivial robustness under transfer attacks and even FPA (better than RMC), but is broken by GMSA-MIN.
    • The authors say ATRM remains robust under their strongest adaptive attack and outperforms standalone adversarial training: on MNIST from 91.61% to 94.32%, and on CIFAR-10 from 41.06% to 53.53%.
    • Cost: ATRM's clean accuracy is lower than Madry et al.'s (MNIST 99.02 vs. 99.16; CIFAR-10 85.11 vs. 86.06).

    Attacking RMC and RMC+

    • RMC (Table 1): FPA can already break RMC paired with a Standard model. Compared with defense-unaware PGD, GMSA-AVG reduces robustness from 98.30% to 0.50% on MNIST and from 97.60% to 8.00% on CIFAR-10. When paired with an adversarially trained model, under GMSA-MIN MNIST drops from 96.10% to 58.80% and CIFAR-10 from 71.70% to 39.60%—the authors note that in this case RMC is actually worse than standalone adversarial training.
    • PGD-skip setting (Table 2, p = 100): the authors confirm the observation of [33]—under PGD-skip, RMC has limited robustness while RMC+ is better, and RMC+ also has some resistance to FPA; but GMSA-MIN breaks both: on MNIST, RMC+(Madry et al.) drops from 69.80% to 4.00%; on CIFAR-10, RMC+(Standard) drops from 60.70% to 4.80%, and RMC+(Madry et al.) drops from 71.70% to 23.50% (the authors note the latter is still worse than standalone adversarial training).

    URejectron's small-perturbation failure mode (Figure 1)

    • Setup: GTSRB + ResNet18, x̃ is 50% normal samples + 50% adversarial samples; the figure gives err (error rate on accepted samples) and rej (rejection rate on normal samples) curves at different rejection-rate thresholds.
    • PGD transfer attack (Figure 1(a)): the authors say URejectron performs as expected.
    • CW transfer attack (Figure 1(b)): the authors say adversarial examples can be constructed that closely resemble clean samples, pass through the discriminator S, and cause a large error rate.
    • Image corruption (Figure 1(c)): the authors say "benign" perturbed samples generated with corruptions such as brightness (the base classifier predicts correctly, accuracy 98.90%) are all rejected by URejectron; the authors acknowledge this is beyond the scope of their guarantee (3), but believe the small-perturbation case needs more careful consideration.

    Other ablations and analyses

    • Iterative process (Figure 2): the attacker evaluates with its own randomness; FPA generally fails to produce attack sets that get stronger round by round; GMSA-MIN is more effective against DANN, and GMSA-AVG is more effective against ATRM.
    • Effect of private randomness (Appendix D.5): on MNIST, DANN's attacker-side robustness on GMSA-AVG's strongest attack set is only 10.97%, while the defender using private randomness can reach 79.37%; the authors explain this as the attack set failing to defeat the entire model space (some historical models F(i) still have 71.28% robustness).
    • Multi-seed repetition (Figure 3): the authors say DANN's and ATRM's robustness does not vary much across different random seeds.
    • Pretrained models (Table 4): on MNIST, the PGD robustness of Standard/Madry et al. is 0.00/93.50 respectively; on CIFAR-10, 0.00/46.80 respectively.
  5. What are some points that can be further explored?

    The authors point out that ATRM is computationally expensive and that the results are preliminary, and state that the separation theorem can be generalized to a broader class of problems; the experiments cover three datasets.

    Limitations and future directions noted by the authors

    • ATRM is computationally expensive: The authors note the limitations of these preliminary results, such as the high computational cost of ATRM (Introduction and Section 5).
    • Results are preliminary in nature: The authors state that these results are preliminary results and that their limitations should be noted (Section 5).
    • Generalization of the separation theorem: The authors state that generalizing the separation result to a broader class of problems is an intriguing direction (Section 5.2).
    • The small-perturbation case needs more careful consideration: After the URejectron experiments, the authors believe that the small-perturbation regime needs more careful consideration (Section 3.2).
    • Further exploring the utility of transductive learning: The authors state that the positive results of ATRM indicate that the utility of transductive learning for adversarial robustness is worth further exploration (Section 6.2).

    Scope of experiments

    • The defenses tested are RMC, RMC+, DANN, ATRM and URejectron, 5 in total, paired with two pretrained models, Standard and Madry et al. (Section 6, Appendix D).
    • The datasets are MNIST, CIFAR-10 (main experiments) and GTSRB (only the URejectron experiments, using a 10-class subset, ResNet18) (Appendix D.2).
    • The attacks are L∞ PGD and its variants (FPA, GMSA-AVG, GMSA-MIN, PGD-skip) and L2 CW (only the URejectron experiments); the perturbation budgets are ε = 0.3 for MNIST and ε = 8/255 for CIFAR-10 and GTSRB (Appendix D.3, D.4).
    • The PGD-skip experiments were independently repeated 1000 times; DANN/ATRM were repeated with 5 random seeds (Appendix D.3, D.6).
    • The paper does not report the number of queries for the attacks, the time per attack, or specific training cost figures for ATRM.
  6. Summarize the main content of the paper

    Formalize the transductive threat model and propose GMSA to break existing defenses, while giving ATRM and a separation theorem as positive evidence.

    An attack+analysis paper on transductive adversarial robustness: the defender dynamically retrains the model at test time based on (possibly perturbed) unlabeled inputs, and the authors systematically examine whether this paradigm can truly improve robustness.

    • Problem: Prior transductive defenses lacked a unified threat model, the attacks used in evaluation were weak (e.g., the non-adaptive attack used by RMC), and existing attack techniques such as BPDA are not applicable to a Γ that truly performs learning.
    • Method: The authors formalize the transductive threat model (Definition 1, white-box, the attacker does not know the defender's private randomness), point out that the attack is a bilevel optimization; leveraging adversarial example transferability, they propose the attack-the-model-space principle, instantiated as GMSA (PGD over the set of historical models, T = 9); and they propose the ATRM defense combining adversarial training with DANN-style representation matching.
    • Attack results: GMSA breaks RMC, RMC+ and DANN—on MNIST, RMC(Standard) robustness drops from 98.30% under PGD to 0.50% under GMSA-AVG (Table 1); under the PGD-skip setting, RMC+(Madry et al.) drops from 69.80% to 4.00% (Table 2); on CIFAR-10, DANN drops from 54.29% under transfer attacks to 0.08% under GMSA-MIN (Table 3).
    • URejectron: Under the small-perturbation setting of GTSRB + ResNet18, the authors state that CW transfer attacks can pass through the discriminator and cause large errors, while benign samples with brightness corruption are all rejected (Figure 1).
    • Positive evidence: The authors state that ATRM still outperforms adversarial training alone under their strongest attack (CIFAR-10: 53.53% vs. 41.06%; MNIST: 94.32% vs. 91.61%, Table 3), and they prove a separation theorem between transductive and inductive settings (Theorem 1, a constructive Gaussian data instance).
    • The authors' conclusion: The attack results indicate that there are difficulties in using transductive learning to improve robustness, but ATRM and the separation theorem provide preliminary evidence supporting its utility; the authors also note that ATRM is computationally expensive and that the results are preliminary in nature.

Related papers

Show 3 more papers
Read originalarxiv.org