Decouple to Forget: A Semantic Intermediary Method for Sensitive Information Unlearning in Large Language Models

Decouple to Forget (D2F) is a targeted unlearning method for large language models that separates sensitive facts from non-sensitive content and deletes only the sensitive part to reduce capability loss. It extracts fact-level intermediary representations from the samples to be forgotten, generates counterfactual answers that retain non-sensitive content and remove sensitive facts, and reduces information leakage through preference-based contrastive optimization. On multiple benchmark datasets, D2F balances forgetting and retention better than strong baseline methods.