Rethinking the Verification of Machine Unlearning
Rethinking the Verification of Machine Unlearning
Translation in progress — untranslated text is omitted.
The researchers argue that machine unlearning verification based solely on output behavior can be deceived, and propose instead the RUV verification method, which compares the model's internal representations. Existing widely deployable post-hoc verifiers mainly infer unlearning from predictive behavior, while outputs remain controlled by the model provider; the paper's output-level deception response (ODR) attack can generate responses that pass output verification while preserving the original internal representations. RUV is a gray-box verifier that compares the intermediate forward representations of the original model and the unlearned model, tracking four stages: activation reorganization, representation displacement, neighborhood destruction and loss of manifold support, and uses matched retained data as a control to distinguish responses targeting specific targets from global model drift. Under exact and approximate unlearning settings, ODR's average deception success rates are 97.0% and 97.7%, respectively, RUV's average verification accuracy is 98.9% and 94.9%, respectively, and it rejects 99.5% and 99.8% of the corresponding ODR submissions.
- Machine unlearning requires deployable post-hoc verification, but mainstream verifiers infer unlearning only from outputs returned by the model (accuracy, membership inference, posterior differences), and these outputs are controlled by the audited provider and can be forged.
- The authors first construct the ODR attack: ODR-Head adjusts only the classification head, and ODR-Gate only replays same-class non-member posteriors for registered queries, without changing representation-layer parameters. They then propose the gray box verifier RUV, which compares the intermediate forward representations of the original and unlearned models, and makes a decision using four evidence views — ARS, RS, NPG, MRD — plus retention controls and joint permutation calibration.
- ODR's average deception success rate against Accuracy, LiRA, and TAPE is 97.0% (class-level) and 97.7% (sample-level). RUV achieves average verification accuracy of 98.9%/94.9% across three datasets and six unlearning methods, and rejects 99.5%/99.8% of ODR submissions; responses also rise monotonically with the fraction of the request executed.
- The paper has no dedicated limitations section; the conclusion proposes extending the method to generative and multimodal models. Experiments are limited to CIFAR10/CIFAR100/FaceScrub image classification and 7 CNN/Transformer backbones, and sample-level accuracy drops to 0.806 at a 1% unlearning fraction.