Generating potent poisoning and backdoor attack samples from scratch with guided diffusion
Generating Potent Poisons and Backdoors from Scratch with Guided Diffusion
Translation in progress — untranslated text is omitted.
The researchers propose a guided diffusion poisoning method, GDP, which uses a poisoning objective to weakly guide a diffusion model to synthesize base samples from scratch, then uses them as initialization to run downstream data poisoning or backdoor attacks, greatly surpassing prior best attacks such as Witches' Brew and Sleeper Agent in success rate. On CIFAR-10, injecting 25-50 poisoning images into ResNet-18 (0.05%-0.1% of the training set) is sufficient for an effective attack, and on ImageNet, 50-100 images (0.004%-0.008%) suffice; the attack remains effective under black-box transfer settings and bypasses all 7 defenses tested (Spectral Signatures, DP-SGD, mixup, STRIP, Neural Cleanse, ANP, ABL). Human evaluation shows the generated base samples retain clean labels and high image quality; code is open-sourced at https://github.com/hsouri/GDP。
- Large-scale web-scraped training sets lack human review, and attackers can upload malicious data to poison or implant backdoors. Existing attacks start from randomly selected clean base samples and perturb them; the authors argue that carefully selected base samples can produce stronger poisons.
- A diffusion model is used to synthesize base samples from scratch under weak guidance from the classifier loss and the poisoning/backdoor gradient-matching loss; these are then used to initialize existing poisoning or backdoor attacks, and finally the poisoned samples are filtered by poisoning loss, keeping those with the lowest loss.
- The authors report that GDP exceeds baselines such as Witches' Brew and Sleeper Agent on targeted poisoning and backdoor attacks on CIFAR-10 and ImageNet; 50 poisoned samples achieve 70% targeted poisoning on CIFAR-10, and it bypasses the 7 defenses tested, and black-box transfer also outperforms the baselines.
- Limitations listed by the authors: a diffusion model must be trained on a specific data distribution, and the computational cost is high; the entire training set or a large subset of it is needed; far more poisoned samples are generated than are actually injected, and the filtering process is inefficient. The experiments cover only CIFAR-10 and ImageNet image classification.
The paper publicly released the method and code for synthesizing poisoning base samples with guided diffusion, showing that a very small poisoning budget can bypass seven defenses; teams that train models on web-crawled data can use it to assess poisoning risk.
Related papers
- Poisoning and Backdooring Contrastive LearningAttacksarXiv