MUDMAN: Meta-Unlearning with Disruption Masking and Normalization for Robust LLM Unlearning

1. 01What problem does this paper attempt to solve?

   After unlearning, dangerous capabilities are easily restored by supervised fine-tuning; the authors use MUDMAN to increase irreversibility.

   How to make forgotten dangerous capabilities difficult to recover after supervised fine-tuning relearning.

   • Occurrence scenario: The authors believe pretraining brings in knowledge needed for manipulation, hacking, and even bioweapon manufacturing, and models may also know the protections used to control themselves. DPO and RLHF often only slightly modify weights to hide knowledge, and jailbreaks or accidental inputs can bring it back.
   • Existing shortcomings: The authors state that dedicated unlearning methods are also easily reversed. Which components can make harmful capabilities truly irreversible, the paper argues, remains unclear.
   • Core observation: In small-scale experiments, most methods can raise forget loss during the unlearning phase, but subsequent supervised fine-tuning usually immediately restores it. The authors therefore treat post-relearning forget loss, or WMDP accuracy, as the quantity to optimize.
   • This paper's approach: Propose Disruption Masking, keeping only unlearning updates with the same sign as retain gradients, and combine with meta-learning and unlearning gradient normalization into MUDMAN. The authors state it outperforms adapted TAR on multiple models and tasks, with the abstract stating 40% higher.

   Collapse this answer

2. 02What related research exists?

   The paper discusses activation interventions, weight ablation, MAML-style unlearning, and that existing unlearning can be reversed in multiple ways.

   The paper classifies existing unlearning by mechanism and emphasizes that they can still be re-elicited.

   • Activation and weight interventions: Zou et al. (2024), Li et al. (2024), and Rosati et al. (2024) interfere with intermediate activations. Wang et al. (2024), Wu et al. (2023), Uppaal et al. (2024), and Suau et al. (2024) locate and ablate weights related to unwanted behavior. The authors use Lo et al. (2024) to show that even when unwanted concepts are directly removed, models can quickly re-represent them with semantically similar neurons.
   • Meta-learning unlearning: MAML from Finn et al. (2017) trains on a copy before taking gradients. TAR (Tamirisa et al., 2024a), Henderson et al. (2023), and Tamirisa et al. (2024b) use it to anticipate how attackers might relearn target capabilities. The authors claim that despite this, no one has yet given a mechanistic explanation for why MAML helps robust unlearning.
   • Reversibility: The authors cite Lee et al. (2024) to say that DPO and RLHF do not remove dangerous knowledge but hide it with minimal weight changes. Lynch et al. (2024) and Łucki et al. (2025) are used to list ways supposedly removed capabilities can still be elicited: jailbreak prompts, few-shot prompts, fine-tuning, in-context learning, out-of-distribution inputs, and disabling refusal with representation engineering. The introduction also uses Zou et al. (2023), Qi et al. (2023), and Deeb and Roger (2024) to show that jailbreaks, fine-tuning, or unexpected inputs can bring capabilities back.
   • Baselines and data: The comparison baseline is adapted TAR. Data are CodeSearchNet, Wikitext, Pile-Bio, Pile, WMDP-Bio, and MMLU; the appendix also uses BeaverTails.

   The authors position this work after these: systematically testing which components make behavior removal harder to reverse, combining Disruption Masking, meta-learning, and gradient normalization into MUDMAN, and comparing against adapted TAR.

   Collapse this answer

3. 03How does the paper address this problem?

   MUDMAN takes unlearning gradients on the adversary, normalizes them, and keeps only updates that share the sign with the retain gradient.

   MUDMAN uses a single loop to connect three things: periodically forking an adversary from the main model, training the adversary on the forget set, and then using the forgetting gradient on the adversary to update the main model. Before applying, gradient normalization is done first, and Disruption Masking is used to discard components whose signs are opposite to the retain gradient. Retain updates and forgetting updates are interleaved.

   Tasks and Constraints

   • Two types of forgetting: For skill forgetting, Python functions from CodeSearchNet are used, with comments and docstrings removed, and retain is Wikitext. The authors say programming is not harmful behavior, but the loss signal-to-noise ratio is high, which is used to compare methods. For knowledge forgetting, the forget set is Pile-Bio, and retain is the rest of Pile.
   • What is measured: The effect on Pile-Bio is measured with WMDP-Bio, a dataset with 1273 multiple-choice questions, answered with temperature=0. Retained performance in this set of experiments uses MMLU, with temperature=1, and the forgetting process is allowed at most a 1 percentage point drop in accuracy.
   • Relearning: Each trial first forgets, then does fixed supervised fine-tuning. The authors say fine-tuning is the most direct and reliable way to re-elicit removed behavior, so the experiments focus on this, rather than jailbreaks or out-of-distribution inputs.
   • Rejection rule: Optuna maximizes the forget loss after relearning; the WMDP experiments instead target accuracy. Trials where retain loss exceeds the initial value plus 0.05 are terminated and rejected. Appendix G.2 changes this in the WMDP search to: when exceeding that threshold, first pause the forgetting updates, and only reject when exceeding the initial value plus 0.1.

   Meta-learning

   • Adversary: A copy of the main model, called the adversary, is trained on the forget set. The forgetting gradient is computed from this additionally trained copy, and then applied to the main model.
   • The authors' hypothesis: Naive forgetting only puts harmful circuits to sleep; backpropagation cannot find sleeping circuits and therefore cannot continue erasing them; after the adversary reactivates these circuits, backpropagation can continue erasing. The paper gives no measurement of this mechanism.
   • Difference from traditional MAML: The traditional approach uses an inner loop to train multiple adversaries and accumulate gradients. The authors say that in early experiments, interleaving updates to the adversary and the main model worked better, so they switched to a single loop and trained one adversary deeply, rather than training multiple shallowly.
   • What is done in one step: Re-forking happens every fork_every_n_loops. Retain and forget batches are sampled; the retain gradient is written to an accumulator with momentum μ and updates the main model; the adversary is updated on the forget set with cross-entropy; then the gradient of the forgetting loss is computed.

   Disruption Masking

   • Rule: A weight's forgetting gradient is kept only if it has the same sign as the retain gradient; otherwise it is set to zero. That is, g_final=g_u (when sign(g_u)=sign(g_r)), otherwise the forgetting update for that weight is 0. The authors say same-sign updates both avoid disruption and passively improve retain.
   • The authors' stated rationale: The model has been sufficiently pretrained, and the weights are close to optimal; unnecessary modifications will disrupt learned representations, and restoring these representations requires substantial computation. Existing methods train on retain while forgetting and repair afterward; here the change is to avoid causing disruption in the first place.
   • Accumulator: What is compared is not the sign of a single-batch retain gradient, but the sign of the SGD momentum accumulator, to look beyond the current batch. The authors say this does not add extra memory, because both types of gradients already need to be stored; it just changes to storing the accumulator.
   • Figure 2: The left is conventional forgetting, the right is masked. The authors say the mask filters out updates that harm retain, thereby raising forget loss without affecting retain loss. The figure text labels retain loss, forget loss, and Step (200, 400); the main text gives no curve readings.

   Normalization, Modules, and Implementation

   • Normalization: In later stages, gradient norms become smaller, and forgetting slows down. Each forgetting gradient is divided by the global norm to keep the strength constant. This norm is the sum of the squared L2 norms of the forgetting gradients of each module, raised to the 0.5 power. The authors say this also makes training easier to tune. Appendix C compares variants such as per-parameter, pre-mask, and post-mask, and adopts global normalization before masking, because it is simpler to implement.
   • Modules: In Figure 5, the authors say the second-layer weights of the MLP and the Q and K of attention noticeably disrupt general performance, are less important for forgetting, and are better omitted. The most effective is the first MLP layer; for gated MLPs, it is the gating matrix. The authors think this may be because only these modules can shut down MLP neurons; once a neuron is inactive, neither the incoming nor outgoing weights can be updated by backpropagation. The main experiments therefore focus on the first layer of each MLP.
   • Specific layer selection: Llama-3.2-1B and SmolLM-135M intervene on gate_proj; pythia-14m's two-layer MLP intervenes on up_proj. Appendix B says this matches the layer whose output enters the activation function. Freezing the remaining modules also reduces the storage needed for gradients, adversary weights, and accumulators. Appendix F writes that when only intervening on gate_proj, the total footprint is the model size plus three times the size of the intervened parameters, which is lower than conventional SGD training; each round reuses the forget forward and backward for a total of 5 forward or backward passes.
   • Losses and baselines: Figure 3 uses negative cross-entropy, which the authors say performs best; Figure 4 uses negative entropy. The authors say these components can be used with any forgetting loss. The TAR baseline is changed to the same single loop, with the initial representation noising removed. Appendix G.1 states that the version that produced Figure 3 also had an adversary decay that was later removed from Algorithm 1.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Using Optuna to compare post-relearning loss on three models; Figure 3's per-bar exact values are not given in the text.

   Experimental setup

   • Models: pythia-14m, SmolLM-135M, Llama-3.2-1B. The authors first iterate quickly on pythia-14m, then validate on the latter two models.
   • Data: Python from CodeSearchNet, retain is Wikitext; knowledge unlearning is Pile-Bio, retain is the rest of Pile. WMDP-Bio has 1273 questions. Appendix D additionally uses animal abuse Q&A from BeaverTails for benign responses.
   • Comparisons: Adapted TAR, and a level without unlearning but with the same relearning. Ablations turn off meta-learning, masking, or normalization one at a time.
   • Metrics and rejection: Main metric is post-relearning forget loss; WMDP experiments look at post-relearning WMDP-Bio accuracy. Retain loss exceeding initial value plus 0.05 is rejected. WMDP experiments additionally require MMLU drop no more than 1 percentage point; answer temperature is 0 for WMDP and 1 for MMLU.
   • Search: Each bar corresponds to one Optuna search, which the authors state has been confirmed to converge and not hit boundaries. Table 2: Llama has 500 trials, unlearning/relearning 120 steps each, about 7 hours; Smol has 500, 300/300, about 9 hours; pythia has 800, 300/300, about 4 hours; WMDP has 300, 480/240, about 20 hours. One step counts as a forward or backward pass. Hardware is Nvidia L40. One method comparison in the main text runs 5 searches.
   • Relearning and aggregation: Relearning is fixed as SGD, learning rate 1e-3, WMDP 3e-4. The authors tried LoRA relearning, stating that initialization luck makes results unpredictable, so they do not use it. Figure 3 takes the mean and standard error of the last 30 valid trials in that search; Figure 4 takes the last 60. The paper does not report repetitions over independent random seeds.

   Main results

   Exact numbers written in the text are as follows. Figure 3 and Figure 4 method bars have no per-bar readings, so the table does not fill them.

   Swipe horizontally to read the table

   Condition	Model and data	Metric	Value	Source
   No unlearning, with same relearning	Llama-3.2-1B, Pile-Bio	WMDP-Bio accuracy	45.5%	Figure 4
   No normalization	pythia-14m, Python	Post-relearning forget loss	4.80±0.14	Table 1
   Per Parameter	pythia-14m, Python	Post-relearning forget loss	7.61±0.48	Table 1
   Global Post Mask	pythia-14m, Python	Post-relearning forget loss	7.60±0.49	Table 1
   Global Pre Mask	pythia-14m, Python	Post-relearning forget loss	8.51±0.55	Table 1

   Table 1's protocol is MUDMAN, 600 unlearning passes, 300 relearning passes, Optuna 600 trials, then take the mean and standard error of the last 50 completed trials. The authors treat higher forget loss as better.

   • Relative to TAR: The authors state that in Figure 3, across three models on Python and Pile-Bio, MUDMAN's post-relearning forget loss is higher than adapted TAR; Disruption Masking accounts for most of the improvement over TAR. The abstract states 40% higher overall, and the text does not write how this percentage is computed from each bar. Figure 3's MUDMAN also includes adversary decay later removed.
   • Ablations: The authors state that meta-learning and normalization help, but not in every setting; sometimes turning them off is better, and the authors state they do not substantially exceed the full method. Figure 4 uses only Llama, negative entropy: the authors state masking is again the main part relative to TAR, and meta-learning and normalization also show clear improvements. The x-axis scale is 41 to 45; except for the 45.5% baseline, each bar's accuracy is not given in the text. Smaller models are not plotted because they are near random.
   • Authors' summary: Section 4.5 states that no ablation consistently exceeds full MUDMAN across settings. The conclusion states that all three components improve robustness of harmful knowledge unlearning, with small overhead, and can be used with any unlearning loss. This coexists with Figure 3's statement that not every setting helps, and the paper does no additional statistical tests.

   Module selection

   • What was tested: Figure 5 compares different target modules and several module combinations, with tasks including Python on pythia-14m and SmolLM-135M, and cruelty on SmolLM-135M. Baseline is the loss without unlearning but with the same relearning.
   • Authors' interpretation: Gate projection (equivalent to MLP first layer on pythia) helps most consistently; V, O, up are other candidates. Q, K, down disrupt retain greatly, and omitting them is better. pythia's Q, K, V are combined into one module, so they could not be separated. Exact values for each bar are not given in the text.

   Cruelty tendency

   • Setup: Appendix D uses animal abuse Q&A from BeaverTails as forget, benign responses as retain, comparing on SmolLM-135M; Figure 6 also has a pythia-14m Python panel. The cruelty subplot labels relearning learning rates of 1e-4 and 5e-3. The MUDMAN variant in that figure uses selective logit loss, not the main text's cross-entropy.
   • Authors' interpretation: Masking has some benefit; meta-learning has no help at all. The authors think such tendencies may be shallower than skills and knowledge. The largest change comes from switching the unlearning loss to cross-entropy. Circuit breakers plus LoRA have no valid trials. Exact bar values are not given.
   • Related negative results: The authors state that selective logit loss is sometimes better than cross-entropy or entropy loss, but unreliable across datasets. Removing adversary decay in Figure 6's Python hurts greatly, but is unreliable across datasets, so it does not enter Algorithm 1.

   Other ablations and analyses

   • Figure 3's search tunes unlearning_rate, retaining_rate, adv_lr, retain_momentum, adv_decay, fork_every_n_loops; the WMDP search tunes only the first two.
   • Relearning learning rate is fixed at 1e-3, WMDP at 3e-4; LoRA relearning is not used due to unpredictable results.
   • Appendix E on pythia-14m and Python separately tries suppressing relearning gradients, ablating by activation or importance, shrinking only weights or activations, absolute gradient aggregation, percentile masking, multiple LoRA adversaries, unlearning momentum, and weight consensus; the authors state most are worse than the main text components.
   • Locating circuits only once before unlearning starts: the authors state the beginning is comparable to full meta-learning, but precomputed circuits become outdated later. LoRA adversaries are more memory-efficient but worse than full copies, and the authors state they are sometimes still competitive.
   • The optimal proportion for percentile masking under automatic tuning is around 50%, which the authors state is similar to the proportion covered by same-sign masking, so they switch to the simpler sign rule.
   • The paper does not report final MMLU scores for each method, nor per-bar readings for Figures 3, 4, 5, 6.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that only supervised fine-tuning elicitation was tested, and that stacking different unlearning methods was not evaluated.

   Limitations and future directions noted by the authors

   • Elicitation method: This study only uses supervised fine-tuning to elicit unwanted behavior. The authors call it the strongest and most reliable elicitation method, but future work could extend to attacks where the attacker has no weight access, such as jailbreak prompts (Limitations).
   • Removal versus hard-to-recover: To distinguish truly removing a capability from merely making it harder to recover, one could follow Deeb and Roger (2024): fine-tune on another set of non-overlapping facts to find facts that are supposedly forgotten and unknown to the attacker in advance (Limitations).
   • Method stacking: For simplicity, only methods using a single algorithm throughout are evaluated, and TAR has its initial representation noising removed, leaving only the meta-learning core. Future work could study stacking different unlearning methods in sequence to look for synergy (Limitations).
   • Granularity: The authors want to understand what happens at individual logits and adjust the unlearning loss to target only key logits; they also suggest token-level forget loss analysis so that the loss increase is distributed across all unwanted tokens rather than concentrated in a few (Limitations).
   • Appendix A: Annotate importance for each token in the forget set; use interpretability to see what happens when forget loss stays high during relearning and then suddenly drops (which the authors call break through and compare to grokking), and whether the pre-breakthrough model is more resistant to other elicitation if fine-tuning is not allowed; isolate and compare adversary training depth; invert the question so that certain key desired behaviors still occur under out-of-distribution, jailbreak, or fine-tuning, and connect to deliberative alignment (Guan et al., 2025).

   Experimental coverage

   • Models: Experimental models are pythia-14m, SmolLM-135M, and Llama-3.2-1B, 3 in total (Section 3).
   • Tasks: The main comparison includes Python skill unlearning and Pile-Bio knowledge unlearning. WMDP-Bio accuracy is only plotted for Llama-3.2-1B; the authors say smaller models are near random, and the appendix states random guessing is 25% and Llama-3.2-1B is about 45% (Figure 4, Appendix G.2).
   • Elicitation and baseline: Relearning is SGD supervised fine-tuning. The authors state they also care about jailbreaks and out-of-distribution attacks, but the experiments focus on fine-tuning (Section 3). The TAR baseline has representation noising removed (Section 4.5).
   • Search summary: Each bar is the mean and standard error of the last 30 or 60 valid trials from one Optuna search. The paper does not report the number of independent random seed repetitions, nor the final MMLU scores of each method.
   • Appendix: Includes target modules (Figure 5), normalization types (Table 1), cruelty tendencies on BeaverTails (Figure 6), and components not in the main method in Appendix E. The MUDMAN used in Figure 3 includes adversary decay, while Algorithm 1 does not (Appendix G.1).

   Collapse this answer

6. 06Summarize the main content of the paper

   MUDMAN combines meta-learning, same-sign masking, and gradient normalization; the authors claim it is harder to restore by relearning than adapted TAR.

   MUDMAN is a training pipeline aimed at making forgotten skills or harmful knowledge harder to recover via supervised fine-tuning.

   • Problem: The authors argue that DPO and RLHF often only hide dangerous knowledge, and dedicated unlearning methods are also easily reversed. They want to identify which components make removal resistant to relearning.
   • Approach: Fork an adversary from the main model, train it back on the forget set, then update the main model with the adversary's unlearning gradients. Gradients are first normalized by global norm and written only when they share the sign with the retain momentum gradient. The main experiments intervene in the first MLP layer; gated MLPs use gate_proj, and pythia-14m uses up_proj. The Figure 3 version also includes an adversary decay later removed from Algorithm 1.
   • Setup: Models are pythia-14m, SmolLM-135M, and Llama-3.2-1B. Tasks are Python versus Wikitext, and Pile-Bio versus the rest of Pile; on Llama, WMDP-Bio's 1273 questions are also used to look at dangerous biological knowledge, and MMLU is used to limit accuracy drop to within 1 percentage point. The baseline is TAR with representation noising removed, leaving only the meta-learning core. Each method gets one Optuna search, then the mean and standard error of the last valid trials are reported.
   • Results: The authors claim that across three models and two tasks, MUDMAN's post-relearning forget loss is higher than adapted TAR, with masking accounting for most of the improvement; the abstract claims 40% higher, and the main text gives no calculation. Meta-learning and normalization are not better in every setting. With no unlearning but the same relearning, Llama's WMDP-Bio is 45.5% (Figure 4); the main text does not give the accuracy of each method's bars. On pythia-14m and Python, Table 1's Global Pre Mask is 8.51±0.55, and no normalization is 4.80±0.14. In the appendix, meta-learning does not help on cruelty tendencies, which the authors suggest may be shallower.
   • Authors' conclusion: The authors claim that selective updates that avoid destroying retain in advance are key to this line, and that the results bring truly irreversible unlearning closer. They also state that only supervised fine-tuning with weight access has been tested so far.

   Collapse this answer

Related papers