Distinguishable Deletion: Unifying knowledge erasure and refusal in large language model unlearning

The researchers propose the Distinguishable Deletion (D²) paradigm, which unifies training-time knowledge erasure and inference-time refusal for large language model unlearning; the paper's experiments show that its EUA method significantly outperforms existing methods. The paper points out that existing knowledge deletion (KD) substitutes suppression of specific token sequences for complete knowledge removal, resulting in biased deletion; distinguishable refusal (DR) may allow harmful knowledge to re-emerge because the underlying knowledge remains. D² constrains the response distribution at the latent representation level rather than specific tokens, erasing the target knowledge and distinguishing it from retained knowledge, so that the refusal mechanism can safely and coherently handle unlearned inputs; a companion energy metric quantifies the degree of knowledge presence and the separation between unlearned and retained content, and EUA accordingly performs energy-bounded unlearning during training and applies an energy-based refusal mechanism at inference. Code is open-sourced at https://github.com/Puning97/EUA-for-LLM-Unlearning。