Can Unlearning Withstand Collaborative Scrutiny? Robust Unlearning for LLM Teams Sharing a Model
Does Unlearning Survive Collaboration? Robust Unlearning for Shared-Model LLM Teams
ACU trains a shared model to suppress team message recovery; on TOFU, relative to output supervision, full-trajectory ROUGE-L drops from 41.8% to 19.3%.
- When teams of a shared model collaborate, unlearning may fail: the final answer can be refused, yet incomplete clues can still synthesize the target. The authors argue that constraining only the final answer or a single message is insufficient.
- ACU uses a frozen reader to reconstruct the forgotten target from single messages, pooled messages, and full trajectories, weights by risk, assigns credit via message replacement, and constrains both direct and collaborative utility.
- On TOFU, relative to output supervision, full-trajectory ROUGE-L goes from 41.8% to 19.3%, and factual recovery goes from 34.2% to 18.7%. On WMDP-bio, ACU's forget accuracy is 27.9%, and MMLU is 65.4%.
- The authors note increased training compute, that the fixed audit does not cover adaptive recovery or longer interactions, and that recovery does not equal parameter-level deletion. The experiments include three backbones, a heterogeneous team, held-out readers, and three-seed Full ROUGE.
- Evaluated models
- Llama-3.1-8B-InstructQwen2.5-7B-InstructGemma-2-9B-it
- Benchmarks
- TOFU (forget10/retain90)WMDP-bioMMLU
- Metrics
- ROUGE-L recallfactual coverage Runlearning efficacyMMLUbenign refusal
The paper finds that LLM team collaboration weakens unlearning and proposes the Aggregation-Conditioned Unlearning (ACU) method to reduce the information recovery rate in team messages. The authors show both theoretically and experimentally that controlling only final answers or individual messages does not guarantee that their combination cannot recover the unlearned content. ACU uses reconstruction feedback from individual messages and aggregated messages to train the shared model, prioritizes perspectives with higher recovery risk, and performs replacement-based credit assignment while protecting direct and collaborative utility. On TOFU, compared with output-supervised training, ACU reduces ROUGE-L recall for full-trajectory recovery from 41.8% to 19.3% and fact recovery from 34.2% to 18.7%, with comparable retained task performance; WMDP-bio is used to test native unlearning and retained utility. The authors state that the method generalizes across team sizes and different readers.
Paper analysis
What problem does this paper attempt to solve?
The authors state that collaboration weakens unlearning: controlling the final answer or a single message cannot guarantee that the combination is hard to recover.
When LLM teams with shared weights collaborate, forgotten knowledge may still be recoverable from combinations of messages.
- Setting: The authors state that teams handle complex tasks through comparative perspectives, shared evidence, and merging partial solutions. If forgetting holds only for a single response, incomplete disclosure may still synthesize the forgotten answer.
- Existing gaps: The authors state that gradient updates, NPO, and SimNPO improve the balance between forgetting and retention, and subsequent work strengthens robustness to retraining, adversarial prompts, and repeated queries, but does not address message exchange among teammates after unlearning.
- Core observation: The authors give a theory and report experiments: suppressing the final answer or a single message does not guarantee that the recovery risk after combination is also low. In Figure 1(a), direct questioning is refused, yet format, identity, and domain cues together still expose the target.
- Collaborative setup:
- Multiple proposers answer independently, and one aggregator synthesizes the messages; all roles share the same post-unlearning parameters, and weights are fixed during inference.
- The main team has N=3, for four target-model calls in total. The forget set is query–target pairs, and the retain set is useful question answering.
- Native exposure looks at direct or team final answers; recovery exposure is reconstructed by an independent reader from intermediate messages, with the target given only to the scorer.
- The paper does not use terms such as white-box or black-box to describe this reader.
- Method: Proposes Aggregation-Conditioned Unlearning (ACU), which trains the shared model with reconstruction feedback from individual and pooled messages and protects direct and collaborative utility.
What related research exists?
Related work is divided into robust unlearning, team safety, and team learning; the authors state that message exchange remains an open training problem.
Robust unlearning
- Effectiveness evaluation: Yao et al. (2024), Liu et al. (2024), and Shi et al. (2024) remove specified knowledge via parameter updates and evaluate with forgetting, retained utility, and privacy. Bărbulescu and Triantafillou (2024) suppress specified knowledge with gradients; NPO and SimNPO (Zhang et al., 2024a; Fan et al., 2024a) balance forgetting and retention.
- Subsequent use: Fan et al. (2025) and Wang et al. (2025a) target the resurgence of forgetting after retraining; Reisizadeh et al. (2026) and Pan and Wang (2026) address recovery under repeated and multi-turn queries. The authors state that these works do not address message exchange among teammates after unlearning.
Team safety
- Unsafe behavior: Chen et al. (2024) study memory poisoning, Gu et al. (2024) study contagious jailbreak, and Zhang et al. (2024b) and Yu et al. (2024) study agent profiling and communication topology.
- Disclosure and inference-time removal: Patil et al. (2025) discuss compositional disclosure; ALU (Sanyal and Mandal, 2025) addresses inference-time removal, and Agentic Unlearning (Wang et al., 2026) addresses parameter–memory backflow. The authors state that preserving learned forgetting in team message exchange remains an open training problem.
Team learning
- Collective feedback: Lowe et al. (2017) and Yu et al. (2022) use centralized optimization to improve individual policies; Song et al. (2022) and others discuss coordination in networked systems.
- LLM teams: Stronger-MAS (Zhao et al., 2026) learns from task rewards; C3 (Chen et al., 2026) assigns credit via message replacement. The authors state that, to their knowledge, ACU is the first to optimize shared-model unlearning for downstream recovery from collaborative message views.
Baselines and benchmarks
- Baseline methods: GA, WGA, SalUn, DPO, NPO, SimNPO, CEU; supervised controls are Native-only, Direct-message, Semantic-direct, RULE-GradDiff, plus retain-only retraining.
- Benchmarks: TOFU forget10/retain90, WMDP-bio, and MMLU.
The authors position ACU after these works: it follows collective feedback and message replacement but writes multi-view recovery into the shared model's unlearning update.
How does the paper address this problem?
ACU updates the shared LoRA with multi-view reconstruction by a frozen reader, risk weighting, and replacement credit.
ACU performs generate–reconstruct–update on the shared model: it first generates team interactions, then a frozen reader checks what the messages leak, and the recovery score together with the retention task updates the parameters (Figure 2).
Collaborative unlearning setup
- Proposers generate messages according to role prompts, and the aggregator synthesizes an answer over the message set. The protocol fixes prompts, visibility, decoding, and budget. The main team has N=3, and weights are fixed during inference.
- The exposure difference between direct deployment and team deployment comes only from different uses of the same model. Role prompts elicit direct, evidence-oriented, and another perspective; Table D.1 gives the full prompts.
- Proposition 1: it is possible to make the final-answer risk 0 while a fixed reader recovers the target by exact match on a candidate view.
- Proposition 2: with N independent uniformly q-valued coordinates, the best single-item full recovery is q^-(N-1), and after pooling it is 1; the final-answer risk can still be 0. Therefore intermediate messages must be checked, and what the reader can infer from the combination must be evaluated.
Multi-view reconstruction
- Views include single proposals, the joint proposal pool, and Full, which includes the final answer. The reader sees only the query and visible messages; a separate scorer gives a score from 0 to 1, and only the scorer sees the target. Native auditing scores the direct answer or the team final answer directly.
- With three proposers there are seven audits: two native, three singleton, one joint, and one Full. The authors state that Full has the most text but is not necessarily the most exposed input for a fixed reader, so all three access types are kept.
- The candidate model's objective is to minimize the highest average audit risk, plus the retention likelihood. Direct and team retention quality must not fall below the pre-unlearning reference minus the allowed loss.
The objective is written as minθ maxa∈ AB Rf(θ,a)+λr Lret(θ): minimize the worst average recovery risk over the audit family, and anchor the model to useful answers with the retention loss; utility has an additional inequality constraint.
Shared model update
- A pilot batch estimates the average risk of each audit, then softmax weighting is applied with temperature T. The weights correspond to the gradient weights of a smoothed maximum; after the pilot weights are fixed, an independent batch is drawn for the update. Appendix A gives the pilot error bound.
- Credit assignment follows counterfactual credit and C3's message replacement: keep the execution prefix, independently sample replacement messages and regenerate the suffix, and the aggregator sees the altered evidence. The mean of replacement scores serves as the baseline; if the original trajectory is above the baseline, that message is suppressed, and if below, it is reinforced. The reader and scorer are frozen.
- The recovery gradient is combined with retention cross-entropy and direct and team utility penalties. When utility falls below the threshold, the penalty multiplier increases. Proposal views supervise the proposers; the native team final answer and Full also supervise aggregation. Gradients from all roles accumulate into the same model.
Selection and deployment
- The default frozen reader is Qwen2.5-7B-Instruct, greedy once per view, with an output limit of 96 tokens. The trainable part is a shared LoRA with r/α of 16/32, AdamW, and a learning rate of 10^-5. The audit temperature T and retention weight are both 0.1, the number of replacements per action is 2, and the utility tolerance is 0.05.
- The core comparison is 100 updates with validation every 10, and the training comparison includes feedback and validation, capped at 18 GPU-hour. When both utilities fall within five percentage points of the reference, the feasible model with the lowest maximum average audit risk is selected.
- Deployment does not use the reader or replacement and still makes the original four target-model calls. Algorithm 1 is in Appendix C.
What experiments did the paper conduct?
On TOFU, ACU's full-trajectory ROUGE-L is 19.3%, Native-only is 41.8%, and utility is close.
Experimental Setup
- Models: Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Gemma-2-9B; the heterogeneous team's proposers are the three models unlearned separately, and the aggregator is the same unlearned Qwen. The main recovery reader is a frozen Qwen2.5-7B-Instruct; the held-out readers are Mistral-7B-Instruct-v0.3 and Gemma-2-9B-it.
- Data: TOFU forget10/retain90, with 20 forget authors and 180 retain authors, and 200 held-out forget questions; retain90 is 3,000 for training, 300 for validation, 300 for test. WMDP-bio uses 800 training documents and 100 validation documents, yielding 4,000/500 query–target pairs. BR uses 1,000 answerable neighborhood questions.
- Baselines: GA, WGA, SalUn, DPO, NPO, SimNPO, CEU. The supervision controls are Native-only, Direct-message, Semantic-direct, RULE-GradDiff; the deletion reference is retain-only retraining.
- Metrics: ROUGE-L recall; factual coverage R, excluding facts already given in the question. TOFU's UE is the complement of the proportion with ROUGE-L not below 0.30. WMDP's UE is the complement of multiple-choice accuracy, with 25% chance accuracy corresponding to 75% UE. Scores are percentages, and differences are percentage points.
- Evaluation: the main external audit uses one greedy reader per view, with an output limit of 96 tokens. On 800 Full reader answers and 4,000 binary decisions, pre-adjudication agreement is 93.075%, Cohen's κ=0.829; annotation was done by the authors, with assistance from GPT-5.6.
- Repeats and budget: core Full ROUGE is the mean and sample standard deviation over seeds 42/43/44. Factual and content audits, unless noted, use a fixed model at seed 42. Comparisons include feedback and validation, with an upper limit of 18 GPU-hour.
Main Results
Team-final unlearning ROUGE (%), per Table 1. Lower means lower exposure in the native team's final answer.
Swipe horizontally to read the table
Team configuration ACU SimNPO GA Llama-3.1-8B-Instruct 12.5 28.6 18.2 Qwen2.5-7B 15.8 32.5 22.1 Gemma 2-9B 11.8 27.5 17.6 Heterogeneous MAS 18.9 35.4 25.6 - The authors state that across the four configurations, ACU's team-final exposure is 15.7–16.7 pp lower than SimNPO's, and team retain is 1.9–4.8 pp higher.
- GA's team-final is lower than SimNPO's, but on Llama Retain is 39.1% and BR is 22.0%; ACU's Retain is 55.6% and BR is 5.2% (Table 1). Across the four configurations, GA BR is 22.0%, 23.9%, 20.7%, 22.0%, and ACU is 5.2%, 6.3%, 4.9%, 5.8%.
- The direct reference for the heterogeneous team is its Qwen aggregator. The authors write that ACU's native deployment gap is 7.9 pp. The authors state that useful collaboration is retained while unlearning improves.
Trajectory-Level Recovery
- On Llama, with a frozen Qwen reading full trajectories: ACU's Full ROUGE is 19.3%±0.9, Native-only is 41.8%±1.4, Direct-message is 28.7%±1.2, Semantic-direct is 23.1%±1.0, RULE-GradDiff is 32.6%±1.3 (Table F.1). The control referred to in the abstract is called output-supervision training.
- Factual recovery R: ACU is 18.7%, Native-only is 34.2%. Direct and team retain each differ by only 0.2 pp. BR goes from 25.4% to 5.2%. The authors state that this corresponds to selective unlearning rather than a general suppression of useful answers.
- On WMDP-bio, the authors state that ACU's unlearning accuracy is 27.9% (chance level 25%), and MMLU is 65.4%; Table 2's UE is 72.1%. Retain-only retraining's Full ROUGE is 21.4%, R is 19.0%, and team retain is 50.6%±0.3 (Table F.1, recovery at seed 42).
Mechanisms, Transfer, and Exceptions
- With a fixed Qwen reader, prompt, and budget, looking only at the final answer: Native-only is 14.7%, ACU is 14.9%. Adding proposals raises them by 27.1 and 4.4 pp respectively (Figure 5a, Table F.12). The authors state that the same reader can separate models that are similar when looking only at the final answer.
- Pools that are fully recoverable but where no singleton and query-only are: Native-only is 14/200, ACU is 3/200; complementary evidence is 10 cases and 2 cases respectively. Of the 10 complementary-evidence cases, deleting the key clue gives full recovery 1/10, and an equal-length irrelevant deletion gives 9/10 (Table F.7).
- With seven proposers, ACU's Full recovery is 25.0%, and Native-only is 49.4% (Figure 4a). For the Qwen target it is 22.1% versus 45.2% (Table F.2); with a Mistral reader on fixed trajectories it is 21.4% versus 45.0% (Figure 5b). In Table F.9, the factual recovery differences across four matched-call protocols are 10.9, 12.8, 14.7, 15.6 pp. After changing the reader output limit to 64 tokens, retraining's Full is 9.8%, ACU is 14.3%±0.9, and the ordering reverses relative to 21.4% and 19.3% at 96 tokens (Table F.18).
Other Ablations and Analyses
- Removing recovery: TOFU/WMDP-bio UE is 4.5/5.3 pp lower; removing the replacement baseline is 3.5/6.7 pp lower; removing utility protection slightly raises UE, and MMLU is 6.2/6.6 pp lower (Table 2).
- Singleton-only Full recovery is 29.6%; without the replacement baseline it is 20.6%, versus ACU's 19.3% (Appendix F.7).
- At 9 GPU-hour, Full ROUGE: ACU 29.7%, Direct-message 35.0%, Native-only 46.2% (Figure 6). The two controls actually continued training to 18 GPU-hour and reached 26.5% and 37.9%, still higher than ACU's 19.3% (Appendix F.6).
- Full RULE (30 GPU-hour, 500 updates) Full ROUGE is 28.9%±1.2 (Appendix F.4). On 60 paired questions, factual support in the original messages goes from 40.3% to 27.7% (Table F.1).
- Across three paired groups, reconstruction is on average 3.8 pp lower in Full ROUGE and 3.1 pp lower in factual coverage than Semantic-direct. MIA AUC: Native-only 0.61, ACU 0.56, retraining 0.48 (Table F.18).
- In a 40-question diagnostic on the public Llama-3.2-1B-Instruct, Aggregate's ROUGE difference relative to Best role is 0.53 pp, and the automatic factual score is −21.96 pp (Table F.17).
What are some points that can be further explored?
The authors note training overhead, that fixed audits do not include adaptive recovery, and that recovery does not equal parameter-level deletion.
Limitations and future directions noted by the authors
- Reconstruction and replacement rollouts add computation and may limit scalability (Limitations).
- The two benchmarks and fixed audits do not cover adaptive recovery or longer interactions (Limitations).
- Recovery depends on the reader and metrics and cannot establish parameter-level deletion (Limitations).
- The authors plan to explore cheaper feedback and broader evaluation of collaborative unlearning (Limitations).
- The authors state that over-reliance on favorable audit scores may mask residual leakage, and broader evaluation is needed before sensitive deployment (Ethical Considerations).
Scope of experimental coverage
- Main results include Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, Gemma-2-9B, and a heterogeneous team (Table 1).
- TOFU uses forget10/retain90 and 200 held-out forget questions; WMDP-bio reports native multiple-choice UE and MMLU (Table 2). Trajectory-level recovery and paired deletion are mainly run on Llama, seed 42 (Appendix F).
- The default team is 3 proposers plus an aggregator; native UE with 5 and 7 proposers, and Full recovery with 7 proposers, are also reported (Table F.23, Figure 4a).
- The trained reader is a frozen Qwen2.5-7B-Instruct; Mistral-7B-Instruct-v0.3 is also used on fixed trajectories, and a two-prompt, two-protocol suite with Gemma-2-9B-it takes the per-question maximum (Table F.20, Appendix F.10).
- Core Full ROUGE is the mean over three training seeds. The paper does not report significance tests. Deployment keeps four target-model calls; the number of replacements per action on the training side is 2 (Appendix C).
Summarize the main content of the paper
The authors state that ACU reduces team-recoverable leakage, with TOFU full-trajectory ROUGE-L going from 41.8% to 19.3%.
ACU writes downstream recovery of collaborative messages into the shared model's unlearning training.
- The problem is that when the same set of unlearning weights is reused between the proposer and the aggregator, low exposure on the final answer or on a single message does not mean these messages are hard to recover when taken together.
- A frozen reader scores recovery on single messages, aggregates, and full trajectories, weights by risk, and updates the shared LoRA using credit assignment over replacement messages, while constraining direct and team utility. Inference is still four target-model calls.
- On TOFU and Llama, ACU's full-trajectory ROUGE-L is 19.3%, versus 41.8% for the output-supervision control Native-only; factual recovery is 18.7% versus 34.2%. In Table 1, ACU team-final across the four configurations is 12.5%, 15.8%, 11.8%, 18.9%.
- Looking only at the final answer, Native-only and ACU are 14.7% and 14.9%; adding proposals raises them by 27.1 and 4.4 pp respectively. With seven proposers, Full recovery is 25.0% versus 49.4%. On WMDP-bio, ACU's unlearning accuracy is 27.9%, and MMLU is 65.4%.
- The authors state that ACU reduces the evidence recoverable from team messages, that a gap remains under a matched training budget, and that it does not increase inference calls. The authors also write that recovery depends on the reader and the metric, and does not establish parameter-level deletion.