MPU: Toward safe and privacy-preserving knowledge unlearning for large language models

Researchers propose the MPU framework to address the dual constraint in large language model knowledge unlearning that neither server parameters nor the client's forget set may be leaked. On the server side, Pre-Process distributes multiple perturbed reparameterized copies for local unlearning on the client, and Post-Process inverse-reparameterizes and aggregates updates with harmonic denoising. Experiments with seven unlearning algorithms show performance comparable to the noise-free baseline; most algorithms show an average drop well below 1% within 10% noise, and some algorithms are even better under 1% noise.