Protecting sensitive data in diffusion model image synthesis via PAC-Private adaptation

Researchers propose a PAC-Private adaptation method for diffusion models against reconstruction attacks, which better preserves subject identity, generation quality, and downstream classification accuracy than differential privacy under equivalent reconstruction privacy. The method first learns compact components using LoRA or Textual Inversion, then calibrates anisotropic Gaussian noise according to the covariance of the repeated mechanism's outputs, perturbing only once after optimization. The evaluation covers few-shot concept personalization and full-dataset image synthesis.