ALKN: knowledge negation adaptive localization for continual LLM unlearning
Adaptive Localization of Knowledge Negation for Continual LLM Unlearning
AI summary
In the continual unlearning setting on three established LLM unlearning datasets, the continual LLM unlearning method ALKN stably exceeds baseline methods in both unlearning effectiveness and utility retention. Addressing the problem that continual unlearning tasks cause aggravated utility degradation, the method is based on the task vector framework, uses dynamic masks to sparsify training gradients, selectively updates only parameters associated with the target knowledge, and adaptively adjusts unlearning strength according to relationships between tasks.