Skip to content

From facts to personas: interpretable persona unlearning in LLMs based on mixture of experts

From Facts to Personas: Interpretable Role Unlearning in LLMs via Mixture-of-Experts

arXiv: 其他 AI 安全论文· arXiv:2611.19393· NeurIPS 2026· Ruihong Zeng, Puning Yang, Jinghui Zhang, Shen Gao, Xiangliang Zhang, Preslav Nakov, Xiuying ChenMBZUAI; University of Electronic Science and Technology of China; University of Notre DameIndexed by this site · 2 days ago· Originally publishedDefenses· Its place in the scan record →

Translation in progress — untranslated text is omitted.

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

AI summaryResearchers propose the Role-playing Unlearning task and the RoleMoE mixture-of-experts framework, enabling LLMs to selectively unlearn a target role's style and knowledge while retaining the language abilities and behaviors of the remaining roles, to address privacy risks, social bias and unsafe outputs that role imitation may bring; unlike prior unlearning tasks that only delete factual information, this task also requires unlearning persona-specific features. RoleMoE maps different roles to more independent expert subspaces, with a specialization loss encouraging each role to adopt a different expert distribution and a disentanglement loss encouraging experts to form distinct capability subspaces, so that unlearning focuses on the target role's experts and structural interpretability improves; the RoleBench-Unlearn evaluation benchmark is also released. Experiments on multiple backbone LLMs show that RoleMoE can precisely and controllably unlearn the target role while better preserving the language quality, knowledge and role consistency of non-target roles. Code and dataset are publicly available at https://github.com/mbzuai-nlp/ROLEMOE。

Questions
A role-playing LLM's ability to imitate a role's style may bring risks of privacy, bias, and unsafe outputs; existing unlearning methods mainly delete fact-level knowledge and struggle to handle role-level style and behavior distributions.
Methods
Proposes the Role-playing Unlearning task and ROLEMOE: on LoRA-MoE, adds role-context gating, a role specialization loss and an expert disentanglement loss for two-stage training, then concentrates the unlearning target on the target-role experts; and builds the RoleBench-Unlearn evaluation.
Experiments and results
On two backbones and multiple unlearning algorithms, ROLEMOE's ODS is lower than LORAMOE's (Table 1/2); under re-induction attacks the forget-set scores remain low, whereas LORAMOE shows a rebound (Table 3); MMLU accuracy is not worse than the baselines (Table 5).
Limitations and future work
The authors note that the experiments cover only English RoleBench data, and the metrics may not fully capture style removal and residual leakage; the experiments use only 2 7–8B backbones, 15 roles, 100 samples per role, and a default forget set of 2 roles.
Read originalNeurIPS 2026