Auditing Differentially Private Machine Learning: How Private is Private SGD?
Auditing Differentially Private Machine Learning: How Private is Private SGD?
Translation in progress — untranslated text is omitted.
The researchers propose using a novel data poisoning attack to audit Differentially Private SGD, testing whether its privacy protection in practice is better than what its existing best analysis guarantees, and showing that such poisoning attacks correspond to realistically feasible privacy attacks. Whereas prior arXiv work by Ma et al. in 2019 used the connection between differential privacy and data poisoning as a means of defending against data poisoning, the novelty of this paper lies in using it as an auditing tool for understanding the privacy of a specific mechanism. The authors believe that this method of understanding the privacy of concrete implementations of differential privacy algorithms in a quantitative, empirical way can complement and influence analytical research on differential privacy.
- The theoretical ε analysis of DP-SGD may be conservative; in practice the actual privacy level is unknown, and the value of ε directly affects the privacy-utility trade-off in deployment. The authors want to quantify how much better the actual DP-SGD of a specific implementation is than the theoretical guarantee.
- Turn data poisoning attacks into a privacy auditing tool: construct dataset pairs differing by k rows, use backdoor test statistics to distinguish the two training outputs, and compute a lower bound on ε via Monte Carlo and Clopper-Pearson confidence intervals; and propose ClipBKD, which poisons along the minimum-variance direction of the gradient, to resist gradient clipping.
- Evaluated on FMNIST, CIFAR10, and P100 with LR and FNN: the authors state that ClipBKD's εLB is 2.5x–1500x higher than membership inference, on average 3.84x higher than the standard backdoor (FMNIST), and mostly within 12.3x of the theoretical upper bound; they also find that εLB is sensitive to the clipping norm and initialization randomness.
- Future directions proposed by the authors: whether the gap between the lower bound and the upper bound can be further narrowed, whether the clipping norm and initialization randomness can be incorporated into the privacy analysis, and the realism and interpretability of attack instances. The experiments cover only three datasets, two types of models, LR and FNN, and only the DP-SGD implementation of TensorFlow Privacy.