How to implant backdoors into diffusion models?

1. 01What problem does this paper attempt to solve?

   The backdoor attack risk faced by diffusion models had not been studied before; the authors propose BadDiffusion to evaluate this risk.

   Core problem: the risk of diffusion models being implanted with backdoors during the training stage has not yet been studied; the authors aim to evaluate whether such attacks are feasible, how low-cost they are, and whether they can be defended against.

   • Problem scenario: diffusion models are costly to train, developers commonly fine-tune from publicly released checkpoints as a warm start, and an increasing number of applications use the same model as a foundation; the authors argue that if a foundation model is secretly implanted with a backdoor, all downstream tasks will be affected.
   • Gap in existing research: existing backdoor research mainly targets classifiers; the authors state that this paper is the first study on the robustness of diffusion models against backdoor attacks ("the first study").
   • Core observation: unlike classifier backdoors, which only change training data and labels, diffusion models have a unique noising/denoising process; based on this, the authors design an attack that simultaneously tampers with the training data and the forward/reverse diffusion steps.
   • Paper proposes: the backdoor attack framework BadDiffusion, which supports both training from scratch and fine-tuning a clean pretrained model, and evaluates two countermeasures, ANP and inference-time clipping.
   • Threat model (Sec. 3.1, "outsourced training attack"):
     • Attacker's goal: release a backdoored model that simultaneously satisfies high utility (generation quality on clean inputs comparable to the clean model) and high specificity (generating the target image y when the input contains trigger g).
     • Attacker's capability: can modify the training process, including the training loss, training data (injecting self-chosen samples into Dtrain), and hyperparameters; can fine-tune a pretrained model or train from scratch.
     • Victim party: users who download a ready-made diffusion model from a third-party repository and accept the model after verifying utility via metrics such as FID/IS.
     • Victim system: image diffusion models; the paper uses DDPM as the target model.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: diffusion models, backdoor attacks and defenses on classifiers, and backdoors on generative models; the authors state that this paper is the first backdoor study targeting diffusion models.

   Diffusion models

   • DDPM (Ho et al., 2020): the target model of this paper's attack, a diffusion model that models generation as a forward noising and reverse denoising Markov chain, on which many subsequent works are based.
   • Work on accelerated sampling: DDIM (Song et al., 2021), Analytic-DPM (Bao et al., 2022), etc. aim to address the slow sampling of diffusion models; the authors note that such methods, which view the generation process as reverse Brownian motion, may be subject to backdoor attacks.
   • SDE perspective: Song et al. (2021) use stochastic differential equations to describe the diffusion process in a unified way.

   Backdoor attacks and defenses on classifiers

   • BadNets (Gu et al., 2017): a backdoor attack for classification tasks that injects a trigger by modifying training data and labels; the authors note that BadDiffusion differs in that it needs to simultaneously modify the training data and the forward/reverse diffusion steps.
   • Defense work: includes detecting Trojan behavior in trained models (Umamaheswari & Selvakumar, 2022; Wang et al., 2020), trigger inversion (Neural Cleanse, Wang et al., 2019; ANP, Wu & Wang, 2021), and model cleansing (Zhao et al., 2020). Among these, ANP is selected as the defense evaluation target in this paper.

   Backdoor attacks on generative models

   • GAN backdoor (Rawat et al., 2022): explores backdoor attacks and defenses against GANs; the authors note that GANs do not contain a diffusion process and cannot be used in this paper's scenario.
   • Rickrolling the Artist (Struppek et al., 2022): compromises conditional (text-guided) diffusion models by backdooring only the text encoder; the authors note that this method does not change the diffusion model itself, differs from BadDiffusion's manipulation of the diffusion process, and therefore cannot be applied to this paper's scenario or compared.

   Baselines and benchmarks

   • Baselines: the clean pretrained model (0% poisoning rate) and the training-from-scratch (400 epochs) scheme, compared with the fine-tuning (50 epochs) scheme.
   • Benchmarks/datasets: CIFAR10 (32×32) and CelebA-HQ (256×256).

   The authors position this paper as: to their knowledge, the first study exploring the backdoor risk of diffusion models, with an attack targeting the diffusion process itself, differing from both GAN backdoors and methods that only backdoor the text encoder.

   Collapse this answer

3. 03How does the paper address this problem?

   BadDiffusion tampers with the forward diffusion process of DDPM and derives the corresponding posterior and loss, so that noise bearing the trigger is decoded into the target image.

   Overall approach: BadDiffusion modifies the forward diffusion process of DDPM so that trigger-bearing noised samples converge to a distribution centered on the trigger, and derives the corresponding reverse posterior and training loss, so that the model generates the target image when it sees initial noise containing the trigger, while behaving consistently with the clean model on other inputs.

   Problem setting and DDPM review

   • Target model: DDPM. The forward process q(x_1:T|x₀) := ∏_t=1^T q(x_t|x_t−1) gradually adds Gaussian noise according to a variance schedule, ultimately yielding a standard Gaussian distribution; the reverse process learns Gaussian transitions, and the training loss can be simplified to mean alignment, i.e., having the model predict the injected noise.
   • Attack goal: train or fine-tune a diffusion model such that generation quality on clean inputs is comparable to the clean model (high utility), and any input containing trigger g generates the target image y (high specificity, measured by the MSE between the generated result and y).

   Backdoored diffusion process

   • Poisoned sample construction: r = M ⊙ g + (1 − M) ⊙ x, where g is the trigger pattern, M ∈ {0, 1} is a binary mask, and black regions indicate that the corresponding pixels are not modified.
   • Backdoor forward process: q(x′_t|x′₀) := N(x′_t; √(ᾱ_t) x′₀ + (1−√(ᾱ_t)) r, (1−ᾱ_t) I), i.e., mapping the backdoor target distribution to a Gaussian distribution q(x'T) ∼ N(r, I) centered on the poisoned image r.
   • Backdoor posterior: the authors derive the posterior mean corresponding to the backdoored forward process (full derivation in Appendix J), which has an additional correction term related to trigger r compared with the DDPM posterior.

   Training algorithm and loss function

   • Loss function: the dataset D = {Dp, Dc} consists of poisoned samples and clean samples; clean samples use the DDPM noise prediction loss, and poisoned samples use the noise prediction loss with the correction term ρtδt/(1−αt)·r (Equation 10 in the paper).
   • Training procedure (Algorithm 1): split Dp from the training set according to poisoning rate p%, with the rest as Dc; at each step sample x, t, ϵ, and update parameters by gradient descent until convergence.
   • Sampling procedure (Algorithm 2): same as DDPM, except for the initial sample—sampling from N(0, I) generates clean images, and sampling from N(g, I) generates the backdoor target.

   Two training schemes and hyperparameters

   • Fine-tuning: fine-tune all layers of pretrained models from the third-party library diffusers (google/ddpm-cifar10-32, google/ddpm-ema-celebahq-256), using 50 epochs for CIFAR10 and 100 epochs for CelebA-HQ.
   • Training-from-scratch: reinitialize google/ddpm-cifar10-32 and train for 400 epochs.
   • Optimization settings: Adam optimizer, with learning rate 2e-4 for CIFAR10 and 8e-5 for CelebA-HQ; batch size 128 and 64, respectively.

   Defense schemes

   • ANP: transfers Adversarial Neuron Pruning to diffusion models, injecting noise into model weights and measuring detection performance by the MSE between the reconstructed target and the true target; because relative perturbation causes gradient explosion on DDPM, the authors switch to absolute perturbation and use Adam (learning rates 2e−4, 1e−4, 5e−5) instead of the original paper's SGD (Appendix B).
   • Inference-time clipping: at each time step of diffusion sampling, clip the image to the pixel range [-1, 1] (Equation 11 in the paper); the authors state that this is a simple mitigation method discovered by chance.

   Collapse this answer

4. 04What experiments did the paper conduct?

   On CIFAR10 and CelebA-HQ, the authors claim that a 5% poisoning rate is sufficient to implant a backdoor, that fine-tuning is cheaper than training from scratch, and that inference-time clipping can mitigate it.

   Experimental Setup

   • Models tested: pretrained DDPMs released by Google — google/ddpm-cifar10-32 (CIFAR10, 32×32) and google/ddpm-ema-celebahq-256 (CelebA-HQ, 256×256), both from the diffusers library.
   • Attack setup: on CIFAR10, 2 triggers (Grey Box, Stop Sign) × 5 targets (NoShift, Shift, Corner, Shoe, Hat); on CelebA-HQ, 1 proof of concept (Eyeglasses trigger, Cat target) (Table 1).
   • Metrics: specificity uses the MSE between generated backdoor samples and the target image (lower is better, averaged over 10K Gaussian noises containing the trigger); utility uses FID (50K training images vs. 10K samples generated by the model without the trigger, lower is better); SSIM is also reported (trend consistent with MSE).
   • Poisoning rates: multiple levels from 0%–90% on CIFAR10; 20%, 30%, 50% on CelebA-HQ.
   • Repeats: except for training from scratch for 400 epochs and the CelebA-HQ experiments, all experiments were independently repeated 3 times and averaged; hardware was a single Tesla V100 (90GB memory).

   Main Results

   Partial results for the CIFAR10 fine-tuning scheme (50 epochs), according to Table 7:

   Swipe horizontally to read the table

   Trigger	Target	Poisoning rate	FID	MSE
   — (clean model)	—	0%	14.83	1.06e−1 (Corner) / 2.41e−1 (Hat)
   Grey Box	Corner	5%	9.92	5.32e−2
   Grey Box	Hat	5%	8.53	1.58e−1
   Grey Box	Corner	20%	12.86	1.48e−4
   Grey Box	Hat	20%	8.89	1.19e−5
   Grey Box	Corner	90%	55.42	3.85e−6
   Stop Sign	Hat	20%	8.17	4.85e−4
   Stop Sign	Hat	90%	7.77	2.81e−6

   • The authors claim that in all settings there is a range of poisoning rates that keeps the model with both low MSE (high specificity) and FID close to the clean model (high utility), and consider a 5% poisoning rate sufficient under most trigger-target settings.
   • The authors point out that the Grey Box trigger's FID degrades at high poisoning rates (70%, 90%) (e.g., FID 55.42 at 90% for the Corner target), and consider that simple triggers may easily cause training to collapse; Stop Sign's FID is always lower than Grey Box's at every poisoning rate.
   • The authors claim that when the poisoning rate is below 30%, BadDiffusion's FID is even better than the clean pretrained model; the paper also reports that a model fine-tuned on clean CIFAR10 for 50 epochs has an FID of about 28.59 (Table 7 note).

   Fine-tuning vs. Training from Scratch

   • What was tested: CIFAR10, Grey Box trigger, Corner/Hat targets, fine-tuning (50 epochs) vs. training from scratch (400 epochs) (Table 4, Figure 3).
   • Results: at a 5% poisoning rate, fine-tuning FID is 9.92 (Corner) / 8.53 (Hat), and training from scratch is 18.06 / 18.01; for MSE at 5%, fine-tuning is 5.32e−2 / 1.58e−1 and training from scratch is 4.63e−5 / 3.23e−6; at 20% and 50% the MSE of the two is comparable (e.g., at 20%, fine-tuning 1.48e−4 / 1.19e−5, training from scratch 8.71e−6 / 2.30e−6).
   • Authors' interpretation: the authors claim fine-tuning has lower FID and comparable MSE in all settings, making the attack more efficient; Appendix A further reports that at a 20% poisoning rate, 10 epochs are sufficient to synthesize the target Hat.

   High-Resolution Dataset (CelebA-HQ)

   • What was tested: fine-tuning google/ddpm-ema-celebahq-256 (100 epochs), Eyeglasses trigger, Cat target (Figure 4, Figure 12b).
   • Results: clean model (0%) FID 8.43, MSE 3.85e−1; 20% poisoning rate FID 7.43, MSE 3.26e−3; 50% poisoning rate FID 7.51, MSE 1.67e−5.
   • Authors' interpretation: the authors claim a 20% poisoning rate is sufficient to backdoor this pretrained model, and at 50% the FID is about 10% lower than the clean model (the authors' original wording is "10% improvement").

   Defense Evaluation

   • ANP: tested on backdoored models with 5%, 20%, 50% poisoning rates (Table 8, 9, Figure 8, 9). The authors claim a larger perturbation budget usually brings lower reconstruction MSE, but ANP is sensitive to learning rate — when the learning rate is slightly increased from 1e−4 to 2e−4, MSE explodes and reconstruction collapses to black images in some epochs; the authors consider ANP an unstable detection method for backdoored diffusion models.
   • Inference-time clipping: CIFAR10, Grey Box trigger (Table 6, Figure 5). At a 20% poisoning rate with the Corner target, adding clipping raises MSE from 1.48e−4 to 4.97e−2, and FID changes from 12.86 to 12.99; with the Hat target at 20%, MSE rises from 1.19e−5 to 1.09e−1. The authors claim clipping can disable the backdoor (MSE becomes larger) while largely maintaining utility (FID similar), and give a hypothesis in Appendix D: clipping weakens the trigger's influence, with the detailed mechanism left for future work.

   Other Ablations and Analyses

   • Number of fine-tuning epochs (Table 5, Grey Box trigger): 20% poisoning rate, Hat target, MSE 8.42e−2 at 10 epochs and 1.10e−5 at 50 epochs.
   • Advanced samplers (Table 3, Appendix E): after switching to DDIM, DPM-Solver, DPM-Solver++, MSE rises to 1.14e−1–1.31e−1; the authors claim that applying BadDiffusion directly works worse (these samplers abandon DDPM's Markov assumption), but FID (9.32–10.92) is still better than the clean model (13.0–16.3).
   • Trigger size (Table 11, Appendix H): when Grey Box is shrunk, MSE becomes higher (e.g., the NoShift target rises from 7.87e−3 at 18×18 to 5.45e−2 at 4×4); Stop Sign's MSE remains stable across sizes.
   • Inpainting task (Appendix C): three corruptions — Blur, Line, Box — were designed, and 2048 images were evaluated with LPIPS; the authors claim BadDiffusion can still repair images without the trigger and generates the target image when the trigger is present.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors point out that inference-time clipping may not stop adaptive attacks and that ANP is unstable; the experiments cover only DDPM and two image datasets.

   Limitations and future directions noted by the authors

   • Robustness of inference-time clipping: the authors note that this defense likely cannot resist adaptive and more advanced backdoor attacks (Sec. 5 Conclusion).
   • Clipping mechanism unclear: the authors state that they only offer a hypothesis for why inference-time clipping works, and leave detailed mechanism analysis to future work (Appendix D).
   • ANP unstable: the authors consider ANP an unstable Trojan detection method for backdoored diffusion models, and look forward to more research on detecting backdoored diffusion models (Appendix B.3).
   • Adaptation to advanced samplers: the authors believe BadDiffusion can be improved if appropriate correction terms are studied for samplers such as DDIM and DPM-Solver (Appendix E).
   • Misuse risk statement: the authors acknowledge that their findings about weaknesses of diffusion models could be misused, but believe that red-team work can accelerate the development of robust diffusion models (Sec. 5).

   Scope of experiments

   • The models tested are 2 Google pretrained DDPMs (google/ddpm-cifar10-32, google/ddpm-ema-celebahq-256); the target diffusion models are only of the DDPM type (Sec. 3.2).
   • The datasets are two image datasets, CIFAR10 (32×32) and CelebA-HQ (256×256); on CelebA-HQ there is only 1 trigger-target pair (Sec. 4.1).
   • Defense evaluation includes two types, ANP and inference-time clipping (Sec. 4.6); the paper does not test adaptive attacks against inference-time clipping.
   • Except for training from scratch (400 epochs) and the CelebA-HQ experiments, experiments are independently repeated 3 times and averaged (Sec. 4); the paper does not report variance or statistical significance tests for each setting.
   • Utility and specificity evaluation is based on automatic metrics such as FID, MSE, and SSIM; the paper does not report human evaluation.

   Collapse this answer

6. 06Summarize the main content of the paper

   BadDiffusion implants a backdoor into DDPM by tampering with the diffusion process; a low poisoning rate is sufficient, and inference-time clipping can partially mitigate it.

   A study on backdoor attacks against diffusion models. The authors state that this is the first study on the robustness of diffusion models against backdoor attacks.

   • Problem to solve: diffusion models are widely used as foundation models for downstream applications, but the risk of backdoors being implanted during the training stage has not been evaluated; the authors study, in an "outsourced training attack" scenario, whether an attacker can release a diffusion model that maintains generation quality while generating a specified target image when it sees a trigger.
   • Method highlights: BadDiffusion simultaneously modifies the training data and the forward/reverse diffusion process of DDPM—aligning trigger-bearing noised samples to a Gaussian distribution centered on the poisoned image, and deriving the corresponding posterior and loss function; at sampling time, clean images are generated starting from N(0, I) and target images starting from N(g, I). It supports both training from scratch (400 epochs) and fine-tuning a clean pretrained model (50–100 epochs).
   • Main results:
     • On CIFAR10, the authors state that a 5% poisoning rate suffices to implant a backdoor under most trigger-target settings; with a 20% poisoning rate, Grey Box trigger, and Hat target, MSE is as low as 1.19e−5 and FID 8.89 (Table 7).
     • Fine-tuning is more efficient for the attack than training from scratch: at a 5% poisoning rate, fine-tuning gives FID 8.53–9.92, versus 18.01–18.06 for training from scratch (Table 4); at a 20% poisoning rate, 10 epochs suffice to synthesize the target Hat (Appendix A).
     • On CelebA-HQ (256×256), a 20% poisoning rate suffices to backdoor a pretrained model; at 50%, MSE is 1.67e−5 and FID 7.51, better than the clean model's 8.43 (Figure 12b).
     • On defenses: the authors state that ANP is sensitive to learning rate and its detection is unstable; inference-time clipping can raise the MSE of a 20% poisoning-rate model from 1.48e−4 to 4.97e−2 while FID remains essentially unchanged (Table 6).
   • Authors' conclusions and implications: the authors believe the risk posed by BadDiffusion is real and that the attack can be carried out at low cost, and call on the community to pay attention to the potential risks and misuse of diffusion models; they also note that inference-time clipping may not defend against adaptive attacks, and that more robust defenses remain to be studied.

   Collapse this answer

Related papers