DP-λCGD: Efficient Noise Correlation for Differentially Private Model Training

Researchers propose a new algorithm for differentially private model training, DP-λCGD, which improves training accuracy while using essentially the same memory as DP-SGD. Existing methods such as the matrix factorization mechanism may need to store historical noise vectors, incurring large memory overhead; DP-λCGD correlates the noise only with the previous iteration and cancels its adjustable part, regenerating the previous round's noise with a pseudorandom number generator rather than storing it. Experiments show its computational overhead is minimal and its accuracy is better than DP-SGD.