Poisoning and backdoor attacks on contrastive learning

1. 01What problem does this paper attempt to solve?

   The authors state that unfiltered internet-scraped data exposes CLIP-like contrastive learning models to a realistic threat of poisoning and backdoors.

   Core question: can an attacker carry out targeted poisoning and backdoor attacks at extremely low poisoning rates on multimodal contrastive learning models (such as CLIP) trained on unfiltered internet-scraped data.

   • Problem setting: models such as CLIP and ALIGN are trained directly on "noisy" and "uncurated" internet data, with no one reviewing data before it enters the training pipeline; the authors believe the likelihood of malicious samples being mixed in is high.
   • Shortcomings of existing methods: existing poisoning/backdoor attacks mostly target manually curated datasets and are hard to inject in practice; moreover, prior clean-label backdoor attacks on average require poisoning about 1% of training data (Shafahi et al., 2018; Saha et al., 2021), which for CLIP-scale datasets means modifying millions of images, which the authors consider unrealistic.
   • Core observation: models overfit training data, and the train-test gap of contrastive learning models is larger than that of supervised classifiers (Radford et al., 2021), so injecting only a small number of image-text pairs can embed the target image into the target concept class.
   • The paper proposes: adapting existing poisoning and backdoor techniques to multimodal contrastive learning and conducting an empirical evaluation; the authors say no new technical ideas are needed, and the main contribution is showing these attacks are immediately feasible, with required injection amounts orders of magnitude lower than prior work.
   • Threat model:
     • Goal: make the poisoned model behave under attacker control when used as an embedding function, feature extractor, or zero-shot classifier; the paper focuses on attacking the image encoder f (the authors say they also confirmed the text encoder g can be attacked).
     • Knowledge: one caption construction method assumes knowledge of CLIP's 80 zero-shot prompt templates; another only requires retrieving texts containing the target word from the training set.
     • Capability: following assumptions in existing literature, can only inject a small number of samples into the training set, 100–10,000 times fewer than prior supervised attacks; after the model is poisoned, the attacker no longer controls the data or algorithm of downstream fine-tuning tasks.
     • Victim system: CLIP-like multimodal contrastive learning models trained on scraped data such as Conceptual Captions and YFCC.

   Collapse this answer

2. 02What related research exists?

   The paper is positioned at the intersection of two lines of work: poisoning/backdoor attacks and multimodal contrastive learning. The authors claim it is the first to study poisoning and backdoors in the latter.

   Poisoning attacks

   • Biggio et al. (2012), Barreno et al. (2006)—proposed the framework of targeted poisoning attacks against supervised learning; this paper follows their threat model and general pipeline.
   • Koh & Liang (2017), Turner et al. (2019), etc.—poisoning attacks in supervised, unsupervised (Kloft & Laskov, 2010; 2012; Biggio et al., 2013), and semi-supervised (Liu et al., 2020; Carlini, 2021) settings; the paper notes that these attacks usually require injecting samples into curated datasets and are difficult to realize in practice, whereas this paper targets unfiltered datasets.
   • Carlini (2021)—semi-supervised learning poisoning requires a 0.1% poisoning ratio; the paper compares against it and states that the attack in this paper requires a 1000 times lower ratio.

   Backdoor attacks

   • Gu et al. (2017) (BadNets), Chen et al. (2017)—proposed patch-based backdoors: any image overlaid with the trigger pattern is classified as the target label; this paper adopts their standard checkerboard trigger.
   • Shafahi et al. (2018), Saha et al. (2021)—clean-label backdoor attacks, which on average require poisoning about 1% of the training data; the paper notes that the attack in this paper requires orders of magnitude fewer injected samples.

   Contrastive learning and multimodal contrastive learning

   • Chopra et al. (2005), Hadsell et al. (2006), Sohn (2016), Oord et al. (2018)—foundations of contrastive learning: mapping similar objects to nearby positions in the embedding space.
   • Radford et al. (2021) (CLIP), Jia et al. (2021), Tian et al. (2021)—multimodal contrastive models trained on unfiltered internet data; the paper cites their "noisy" and "uncurated" wording to argue the attack surface, and uses the open-source implementation of CLIP as the attack target.

   Baselines and benchmarks: The paper sets no attack-method baseline (the authors state no new technical ideas are needed); the comparison is with the poisoning ratios required by prior attacks (about 1%, 0.1%). The experimental benchmarks are Conceptual Captions-3M and a YFCC 15M subset, with ImageNet for downstream evaluation.

   Authors' positioning: The authors state that this paper is the first work to study poisoning and backdoor attacks on multimodal contrastive learning, and that its contribution is empirically demonstrating that existing attack techniques are immediately feasible in this domain and at lower cost.

   Collapse this answer

3. 03How does the paper address this problem?

   Pair a target image or a trigger-added image with captions related to the target label into image-text pairs and inject them into the training set, using diverse captions to ensure that the image encoder is poisoned.

   Overall approach: following the poisoning framework of Biggio et al. (2012), adapt poisoning/backdoor attacks from supervised learning to multimodal contrastive learning—the attacker only needs to construct malicious image-text pairs and add them to the training set, without modifying the training algorithm.

   Targeted poisoning attack

   • Constructing the poisoning set: given a target image x0 and a target label y0, construct a caption set Y0 related to y0; the poisoning set is P = {(x0, c) : c ∈ caption set}, and the poisoning quantity is controlled by increasing or decreasing the size of the caption set.
   • Bypassing automatic sanitization: the authors state that exact deduplication can be bypassed by adding small Gaussian noise to images, performing synonym substitution on text, or adding irrelevant words, without reducing attack quality; the authors believe that adversarial deduplication detection will always be feasible.
   • Two methods for constructing the caption set: first, retrieve all texts in the training set that contain the target label string (the authors state that most are relevant and a few are misleading, but the attack remains effective); second, assume knowledge of CLIP's 80 zero-shot prompt templates and use or repeat these templates directly.

   Key design: ensuring poisoning lands on the image encoder

   • Problem: the contrastive loss can be minimized by updating the image encoder θ, the text encoder φ, or both jointly, and only the first matches the attack goal.
   • Countermeasure: use a diverse caption set—it is simpler for the model to modify the embedding of a single poisoned image than to modify the embeddings of all texts in the caption set; on this basis the authors believe the image encoder will be poisoned.

   Extension to backdoor attacks

   • Change: poisoned samples no longer fix the same x0, but instead overlay a trigger pattern bd on different images xi: P = {(xi ⊕ bd, c) : c ∈ caption set, xi ∈ Xsubset}.
   • Trigger: a standard checkerboard patch overlaid on the image (the example in Figure 1 is a 16×16 patch); there are two placement modes—consistent (fixed at the top-left corner) and random (random position).
   • Tightened assumption: the backdoor attack no longer uses zero-shot prompt templates and uses only captions retrieved from the training set.

   Evaluation method and stable metric

   • Downstream usage: zero-shot classification (using CLIP prompt templates) and linear probing (training a linear classifier on embeddings of 50,000 randomly sampled ImageNet training images).
   • Success rate: train 32 models and count the proportion of cases where f(x0) is classified as the target label.
   • backdoor z-score: a metric introduced to reduce variance, z = (Mean⟨ f(u⊕ bd), f(v⊕ bd)⟩ − Mean⟨ f(u), f(v)⟩) · Var⟨ f(u), f(v)⟩^−1/2, measuring how anomalous the embedding similarity of two trigger-added images is relative to natural images; the authors state that the pairwise cosine similarity of natural images is about 0.1 and approximately normally distributed, and that the image-pair similarity has an exponential relationship with the probability that both are classified the same (Figure 3), so the z-score can directly measure attack success rate. In the ablation, each data point is derived from 8 trained CLIP models.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Poisoning 3/3M samples gives a 40% zero-shot attack success rate; at a 0.01% poisoning ratio, the backdoor attack success rate is 50%.

   Experimental setup

   • Model under test: an open-source implementation of CLIP (Ilharco et al., 2021; Turgutlu, 2021), ResNet-50 vision model + Transformer language model, following CLIP's default hyperparameters.
   • Training configuration: batch size 1024, 8 V100 GPUs, 30 epochs, learning rate .0002, Momentum SGD, weight decay 0.02; the authors say this implementation's accuracy on CC3M exceeds the value reported by OpenAI.
   • Datasets: Conceptual Captions-3M (3 million samples) and a 15 million sample subset of YFCC (the same subset as the official CLIP implementation); ImageNet is used for downstream evaluation.
   • Metrics and criteria: attack success rate (the proportion of models classifying the target image as the target ImageNet class) and backdoor z-score; poisoning experiments train 32 models, backdoor experiments 32 models per configuration, ablations 8 models per data point.
   • Harmlessness check: the authors say all poisoned/backdoored models show no statistically significant drop in zero-shot test accuracy.

   Main results

   The paper's main results are presented as figures (Figure 2); key numbers given in the text:

   Swipe horizontally to read the table

   Attack	Dataset	Poison amount	Success rate	Source
   Targeted poisoning (zero-shot)	CC3M	3 / 3M (0.0001%)	40%	§4.2
   Targeted poisoning	YFCC 15M	15 / 15M	20%	§4.2
   Backdoor (zero-shot)	CC3M	0.01%	50%	§4.3

   • The authors say poisoning experiments sweep 1–512 poison samples (Figure 2 left), and backdoor experiments are run with 150–1,500 samples at three rates of 0.0005%/0.01%/0.05% (Figure 2 right), with shading of one standard deviation.
   • The authors say that although YFCC is 5 times larger, attacking a YFCC-trained model is not much harder than attacking CC3M.
   • The authors say that compared with semi-supervised poisoning (Carlini (2021), requiring 0.1%), the rate required by this paper's attack is 1000 times lower.

   Embedding similarity analysis (Figure 3)

   • What was measured: the relationship between the embedding similarity of ImageNet validation image pairs and the probability that both have the same true label; and the similarity of trigger-added image pairs after 0.01% poisoning.
   • Results: the authors say the two are exponentially related (Figure 3 left, y-axis on log scale); at 0.01% poisoning, the pairwise similarity of any two trigger-added images reaches 0.78, whereas natural images are typically about 0.1, which the authors say is five standard deviations higher (Figure 3 right).

   Ablation: poisoning rate and patch placement (Figure 4)

   • What was measured: 5 poisoning rates (0.0025%–0.05%, i.e., 75–1,500 samples) × two poisoning placement methods, 8 independent trials each, 80 experiments in total.
   • Results: at 75 samples (0.0025%), consistent poisoning + consistent testing gives a z-score of 2.5, while random poisoning gives a z-score near 0; beyond about 300 poison samples, random placement attacks are stronger and remain unchanged for random-position testing. The authors conclude: below 0.01% poisoning, fixed positions are better; above that rate, random placement is better.

   Ablation: data scale, model scale, and patch size (Figures 5, 6)

   • Dataset scale: with 300 poison samples fixed, success rate stays consistent between 50,000 and 3,000,000 samples; with 75 samples fixed, success rate declines once the dataset exceeds one million (Figure 5 left). The authors say that as long as "enough" is injected, increasing dataset size cannot prevent the attack.
   • Model scale: 1M-parameter models were never successfully backdoored; 5M-parameter models' z-score fluctuates between 0.3 and 5.4; 30M-parameter models' z-score is always 5.1–5.9 (Figure 5 right). The authors say larger (more accurate) models are easier to backdoor.
   • Patch size: at 0.0025% poisoning, 2×2 patches fail, attacks succeed from 4×4, success rate rises statistically significantly up to 16×16, and larger sizes (up to 32×32) show no further improvement and may even decline (Figure 6).

   Exceptions and negative results

   • In poisoning attacks, linear probes and zero-shot are vulnerable simultaneously; but in backdoor attacks the authors observed cases where the zero-shot model was compromised while the linear probe was not, which the authors do not explain and leave for future work.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list three directions: defense research, explaining the phenomenon, and extending to self-supervised learning; the experiments cover only open-source CLIP and two datasets.

   Limitations and future directions noted by the authors

   • Understanding the difference between zero-shot and linear probes: In backdoor attacks, the zero-shot model may already be compromised while the linear probe is not; the authors call deeper understanding of this phenomenon an interesting future direction (§4.3).
   • Defense research: The authors consider developing defenses that filter malicious poisoned samples from the training set a difficult but important future direction, and give the example that in semi-supervised settings poisoning can be detected by monitoring training dynamics (Carlini, 2021) (§6).
   • Extending to broader self-supervised learning: The authors hope future work will extend the analysis of multimodal contrastive learning to self-supervised learning as a whole (§6).
   • Reproduction cost: The authors state that training a single CLIP model takes about 100 GPU hours on CC3M and about 600 GPU hours on YFCC, and that compute resources make the paper difficult to fully reproduce (Reproducibility Statement).

   Scope of experiments

   • The model under test is an open-source implementation of CLIP (ResNet-50 + Transformer); the model-scale ablation covers 1M–30M parameters (§4.1, §5.1.2).
   • The datasets are Conceptual Captions-3M and a YFCC 15M subset; downstream evaluation uses ImageNet zero-shot classification and linear probes (§4.1).
   • For backdoor triggers, only checkerboard patches are evaluated, with sizes from 2×2 to 32×32, placed in either consistent or random manner (§3.3, §5.1.3).
   • Number of repetitions: 32 models for poisoning experiments, 32 models per configuration for backdoor experiments, and 8 models per data point for ablations (§4.2, §4.3, §5.1).
   • The paper does not report evaluation of any defense method; defenses are discussed only as a future direction (§6).

   Collapse this answer

6. 06Summarize the main content of the paper

   The authors state that for contrastive learning models trained on unfiltered internet data, 0.01% poisoning suffices for a backdoor and 0.0001% suffices for targeted poisoning.

   A paper on data poisoning and backdoor attacks against multimodal contrastive learning (ICLR 2022), studying CLIP-like models trained on unfiltered internet-scraped data.

   • Problem: The training data of such models is scraped directly from the internet with no human review, and the authors consider it highly likely that attackers mix in malicious samples; meanwhile, existing backdoor attacks on average require poisoning about 1% of the data, which is unrealistic for datasets of this scale.
   • Method: Pair the target image (or multiple images overlaid with a checkerboard trigger) with captions related to the target label as image-text pairs and inject them into the training set; use a diverse caption set to push the model to modify the image encoder rather than the text encoder; captions can come from retrieval in the training set or from CLIP's 80 zero-shot prompt templates.
   • Result one (poisoning): Poisoning 3/3M samples (0.0001%) on CC3M gives a 40% zero-shot classification attack success rate; poisoning 15 images in YFCC 15M gives a 20% success rate (§4.2).
   • Result two (backdoor): At a 0.01% poisoning ratio, the backdoor attack success rate on the zero-shot classifier is 50%; the embedding similarity of trigger-added image pairs reaches 0.78, versus about 0.1 for natural images (§4.3, Figure 3).
   • Result three (scale): With 300 poisoned samples fixed, the success rate is unchanged as the dataset grows from 50 thousand to 3 million; the 30M-parameter model's z-score stays stable at 5.1–5.9, and the 1M-parameter model was never successfully backdoored (Figure 5).
   • Authors' conclusion: Training on unfiltered data exacerbates the poisoning risk, and increasing dataset scale cannot prevent the attack; the authors consider developing defenses that filter poisoned samples and extending the analysis to broader self-supervised learning important future directions.

   Collapse this answer

Related papers