Interfering with model training using adversarial shortcuts

1. 01What problem does this paper attempt to solve?

   How to modify a public dataset so that it cannot be used to train ML models while preserving the data's original semantics.

   Core question: how to prevent publicly released data from being used without authorization for machine learning training.

   • Scenario and motivation: data is released publicly for diverse purposes (sharing with friends, promoting photographic works), but not all stakeholders agree to its use for ML; the authors mention copyright, deepfakes, facial recognition and other potentially harmful uses.
   • Shortcomings of existing methods: the authors state that traditional privacy protection approaches such as anti-crawling techniques and legal agreements have proven insufficient in recent years to protect user data (citing Hill et al., 2020).
   • Core observation: DNNs often rely on non-robust, local, texture-like features and shortcuts; the authors propose deliberately implanting shortcuts in data to induce models to "lazily" rely on spurious signals rather than semantic features.
   • What the paper proposes: three dataset modification techniques based on adversarial shortcuts (sparse pixel pattern, visible watermark, brightness modulation), as a proof of concept in multi-class image classification settings, compared with concurrent work Fowl et al. (2021a).
   • Threat model (defender's perspective, Section 3):
     • Defender's goal: release a modified training set Dtrain' such that DNNs trained on it have low accuracy on the unmodified Dtest.
     • Constraints: modified images should differ minimally from the originals and preserve semantics (objects, shapes, colors); labels yi are not modified, and the authors assume that a party obtaining the data can reconstruct yi even without labels.
     • Capability boundary: the defender controls only the training data itself, not the model optimization process (the authors state this setting is more challenging than that of Shumailov et al. (2021)).
     • Victim system: image classification DNNs trained from scratch on the modified data.

   Collapse this answer

2. 02What related research exists?

   Related work covers data poisoning, backdoors, DNN shortcut learning, instance hiding, and watermarks; the authors state that their setting does not control the optimization process.

   Parallel work on the same problem

   • Fowl et al. (2021a)—proposes a poisoning method based on second-order gradients, with the same goal of making a dataset unusable for training; the authors compare experiments with it in Section 5 and state that their simpler method is more effective at disrupting training.
   • Shumailov et al. (2021)—disrupts training with a data-ordering attack; the authors note that their setting is more challenging because it does not assume control over the model optimization process.

   Poisoning and backdoors

   • Trojaning/backdooring (Schwarzschild et al., 2020; Goldblum et al., 2020)—the attacker can influence the input at training and inference time, and sometimes also the training process.
   • Data poisoning (Biggio et al., 2011; 2012)—changes the decision boundary by modifying a small number of training samples; targeted poisoning targets specific inference samples, split into the transfer-learning setting (Shafahi et al., 2018; Zhu et al., 2019; Aghakhani et al., 2020) and the training-from-scratch setting (Muñoz-González et al., 2017; 2019; Huang et al., 2020a; Geiping et al., 2020; Huang et al., 2021; Fowl et al., 2021b).

   DNNs' tendency toward shortcut learning

   • Several studies (Zhang et al., 2016; Jo & Bengio, 2017; Madry et al., 2017; Geirhos et al., 2018; Ilyas et al., 2019) find that standard models rely on non-robust, local, texture features; Zech et al. (2018), Badgeley et al. (2019), and DeGrave et al. (2021) find that models exploit confounders or shortcuts. The authors state that this paper does not attribute failure to naturally occurring shortcuts but deliberately introduces shortcuts.

   Instance hiding and watermarks

   • Instance hiding (Huang et al., 2020b; Carlini et al., 2020)—makes data useful to ML but unreadable to humans; the authors state that this paper's goal is the opposite.
   • Image watermarking (Podilchuk & Delp, 2001; Singh & Chadha, 2013; Dekel et al., 2017)—prevents public images from being used without authorization; the authors state that this paper is similar in spirit; among these, the finding of Dekel et al. (2017) that shape changes make watermarks harder to remove is used to design the visible watermark.

   Baselines and benchmarks: The comparison baseline is the protected CIFAR-10 shared by Fowl et al. (2021a) (parameter ε=8/255); the experimental benchmarks are CIFAR-10 and ImageNet.

   The authors position this paper as: using shortcut learning, a phenomenon previously regarded as a harm, as a security measure, with a method simpler than Fowl et al. (2021a).

   Collapse this answer

3. 03How does the paper address this problem?

   Generate a class-specific non-robust signal for each class (pixel pattern, watermark, brightness modulation) to induce the DNN to take shortcuts and fail to generalize.

   Overall idea: add a modification unique to each class k to form an adversarial shortcut, so that the DNN quickly reaches high accuracy on the training set by relying on this simple signal, thereby not learning semantic features and failing to generalize on unmodified data. All three techniques allow adjusting strength, trading off between disrupting training and preserving visual features.

   Problem formalization (Section 3)

   • Dataset Dtrain = {(xi, yi)}, where xi is a w×h×c RGB image and yi ∈ {1,...,K}; the test set Dtest is independent and identically distributed with Dtrain.
   • The goal is to construct a modified dataset Dtrain' = {(x'i, yi)} satisfying two properties: semantic preservation (minimal modification, ideally visually indistinguishable, labels unchanged); and low accuracy on Dtest for models trained on Dtrain'.

   Sparse pixel pattern (Section 4.1)

   • For each class k, generate a random binary mask ∆k ∈ {0,1}^(w×h×c): each value is sampled δ ~ N(µ, σ), and δ is set to 1 if it exceeds the midpoint of the pixel brightness range (0.5 when normalized to 0-1); σ is fixed at 0.2, and µ is tuned in experiments.
   • The modification formula is x'_i = (1 - \Delta_k) \odot x_i + \Delta_k \cdot x_{max}, i.e., pixels indicated by the mask are set to maximum brightness xmax.

   Visible watermark (Section 4.2)

   • Use MNIST digits to spell out the class index as a class-specific watermark: for CIFAR-10, one corresponding digit is randomly sampled per class; for ImageNet's 1000 classes, at most three random digits are used; the authors state that shape variation makes the watermark harder to remove (citing Dekel et al., 2017).
   • The binary watermark image M is blended with the original image by parameter α ∈ [0,1]: x'_i = \alpha \cdot M + (1-\alpha) \cdot M \odot x_i + (1-M) \odot x_i; α=0 has no effect, α=1 fully covers.

   Brightness modulation (Section 4.3)

   • For each class, randomly select square centers and brighten or darken with equal probability: given γ ∈ [0.5,1], darkening multiplies by γ and brightening multiplies by 2−γ; repeat T times, with squares allowed to overlap, forming a checkerboard-like pattern.
   • This is equivalent to sampling a class-specific mask Bk and modifying according to x'i = Bk ⊙ xi; for ImageNet experiments T=600, and for CIFAR-10 T=32. The authors state that this design is harder to bypass with basic countermeasures than the previous two.

   Evaluation method (Section 5, Appendix A)

   • Only the training set is modified; the validation set remains unchanged; protection success is measured by the best achievable accuracy on the clean validation set.
   • CIFAR-10: train for 50 epochs with batch size 1024, sweeping learning rates {0.1, 0.01, 0.001, 0.0001}, architectures {ResNet18, DenseNet201, VGG11, SqueezeNet} and 4 random seeds, reporting the best accuracy across all runs.
   • ImageNet: use the official PyTorch training script and ResNet18; the authors state that due to compute constraints only default hyperparameters were used.
   • Countermeasure tests: aggressive data augmentation (random crop 28×28, horizontal flip, affine transformation, color jitter) and adding Gaussian noise (mean 0, σ=0.05).

   Collapse this answer

4. 04What experiments did the paper conduct?

   Training ResNet18 and other models on CIFAR-10 and ImageNet, all three modifications substantially reduce clean test accuracy, and two types of countermeasures were tested.

   Experimental setup

   • Models tested: ResNet18, DenseNet201, VGG11, SqueezeNet (CIFAR-10, Appendix A); for ImageNet only ResNet18. All are torchvision implementations trained from scratch.
   • Datasets: CIFAR-10 and ImageNet; only the training set is modified, and the validation set remains clean (Appendix A).
   • Baselines: unmodified dataset; protected CIFAR-10 shared by Fowl et al. (2021a) (ε=8/255; the authors note that other parameters may be stronger).
   • Metric: best achievable accuracy on the clean validation set; for CIFAR-10, the best over hyperparameters and seeds within 50 epochs.
   • Number of repetitions: CIFAR-10 sweeps 4 learning rates × 4 architectures × 4 seeds; ImageNet uses only default hyperparameters (the authors cite limited compute).
   • Sample size: the paper does not report the number of samples used for each dataset.

   Main results

   According to Figure 3 (CIFAR-10, ResNet18, best clean test accuracy after 50 epochs) and Figure 4 (ImageNet, ResNet18):

   Condition	CIFAR-10	ImageNet
   Unmodified	above 70%	above 50%
   Pixel-based µ=0.01	no more than 40%	below 30%
   Visible watermark α=0.5	below 40%	below 30%
   Brightness mod. γ=0.9	below 40%	not reported
   Fowl et al. (2021a)	at most 60%	not reported

   • The authors state that the pixel-based modification with µ=0.01 changes on average only 22 of 3,072 pixels per image, yet "almost halves" accuracy.
   • The authors state that compared with Fowl et al. (2021a), their method suppresses accuracy further (60% vs below 40%, see also Figure 9), and that their simpler method is more effective at disrupting training.
   • The authors state that all modifications affect model performance even under the weakest settings tested.

   Training dynamics analysis (Figure 1)

   • What was measured: training and validation accuracy curves under pixel-based (µ=0.01) protection on ImageNet.
   • Results: models can reach 90% accuracy on the protected training set, but validation accuracy stalls; on unmodified data validation accuracy can reach 70%, with no such generalization gap.
   • Authors' interpretation: the model fits class-specific simple patterns rather than true semantics, indicating that the goal of disrupting training was achieved.

   Countermeasure stability (end of Section 5.1, Figures 10–13)

   • What was measured: pixel-based plus Gaussian noise (Figure 10), pixel-based and watermark plus aggressive data augmentation (Figures 11, 12), brightness modulation plus aggressive augmentation (Figure 13).
   • Results: for pixel-based, the best achievable accuracy is unchanged under both types of countermeasures; brightness modulation (γ=0.9) can be undone by aggressive augmentation, and the model recovers effective training.
   • Authors' recommendation: use a stronger γ=0.70 for brightness modulation, making the modification more obvious.

   Other ablations and analyses

   • Figure 5: validation accuracy under visible watermark protection with different α on ImageNet (the main text states below 30% at α=0.5).
   • Figures 6–8: best achievable accuracy on CIFAR-10 for the three methods under different strength parameters (µ, α, γ); µ=0, α=0, γ=1.0 correspond to unmodified data.
   • Figure 9: best achievable accuracy of the Fowl et al. (2021a) method under different settings.
   • Figures 14–16: example CIFAR-10 perturbation images for the three modifications under different parameters.
   • Exception: the authors acknowledge that SOTA training accuracy was not reached on either CIFAR-10 or ImageNet, attributing this to compute constraints or insufficient hyperparameter tuning, and state that a unified training pipeline ensures fair comparison.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that the work covers only multi-class image classification; future directions include stronger modifications, attempts to undo the protection, and extension to other tasks.

   Limitations and future directions noted by the authors

   • The work focuses on the narrow setting of multi-class image classification (Discussion section).
   • Future direction: study more effective dataset modification methods (Discussion section).
   • Future direction: attempt to undo the protective modifications (Discussion section).
   • Future direction: develop new methods for other ML tasks, e.g., preventing unauthorized development of deepfakes or face recognition systems (Discussion section).
   • The authors state that SOTA training accuracy is not reached on CIFAR-10 and ImageNet, possibly due to compute limits or insufficient hyperparameter tuning (Section 5).
   • The authors state that, due to limited compute, ImageNet experiments are trained only with default hyperparameters (Appendix A).

   Scope of experiments

   • The models tested are ResNet18, DenseNet201, VGG11, and SqueezeNet, 4 in total (CIFAR-10); for ImageNet only ResNet18 (Appendix A).
   • Countermeasure evaluation includes two types: aggressive training-set augmentation and added Gaussian noise (σ=0.05), and is tested only for some methods (Section 5.1, Figures 10–13).
   • The only external method compared is Fowl et al. (2021a), using their shared ε=8/255 version; the authors note that other parameters may achieve stronger training disruption (Section 5.1 footnote).
   • The paper does not report the number of dataset samples used in each experiment.
   • The paper does not report a quantitative or human evaluation of how discernible the modifications are to the human eye (only example images in Figure 2 and Figures 14–16).

   Collapse this answer

6. 06Summarize the main content of the paper

   Proposes the adversarial shortcuts data protection method; three class-specific modifications make the DNN take shortcuts and fail to generalize on clean data.

   A proof-of-concept work on preventing publicly released data from being used without authorization for ML training, targeting the multi-class image classification setting.

   • Problem: Publicly released data may be used to train models without consent (e.g., face recognition, deepfakes); the authors state that anti-scraping and legal agreements have proven insufficient.
   • Method: Proposes adversarial shortcuts—adding a unique non-robust signal to each class to induce the DNN to rely on shortcuts rather than semantic features. Three techniques: sparse pixel patterns (controlled by µ, masked pixels set to maximum brightness), visible watermarks (MNIST digits spelling the class index, visibility controlled by α), and brightness modulation (controlled by γ, checkerboard light and dark squares).
   • Result 1: On CIFAR-10, the best clean test accuracy of ResNet18 drops from above 70% to below 40%; for pixel-based with µ=0.01, on average only 22/3,072 pixels are changed per image (Figure 3).
   • Result 2: On ImageNet, pixel-based (µ=0.01) and visible watermark (α=0.5) protection bring the best validation accuracy below 30%, versus above 50% when unmodified (Figure 4).
   • Result 3: Compared with the protected CIFAR-10 of Fowl et al. (2021a), whose models still reach 60% accuracy, the authors state that their simpler method disrupts better.
   • Result 4: Pixel-based modifications remain stable against aggressive data augmentation and Gaussian noise; brightness modulation (γ=0.9) can be undone by aggressive augmentation, and the authors recommend using γ=0.70 instead.
   • Authors' conclusion: Simple patterns can disrupt DNN training with little effect on image semantics; future work can explore more effective modifications, attempts to undo the protection, and extension to other tasks such as deepfakes and face recognition.

   Collapse this answer

Related papers