Turning black boxes into white boxes: Privacy leakage caused by dataset distillation

The researchers propose an attack method called Information Revelation Attack (IRA) targeting dataset distillation, which can predict the distillation algorithm and model architecture used, and infer the membership of training samples and reconstruct sensitive samples from the real dataset. The authors point out that although synthetic datasets are regarded as privacy-preserving, they implicitly encode the weight trajectory of the distilled model, and are thus information-excessive and exploitable by attackers. The study validates this leakage risk on multiple advanced distillation methods.