Per-Parameter Task Arithmetic for Large Language Model Unlearning

Researchers propose per-parameter task arithmetic (PerTA), which mitigates over-forgetting in large language model unlearning by scaling task vectors according to parameter importance. The method estimates each parameter's importance relative to forgetting and retention, and computes scaling weights using gradients (PerTA-grad) or a diagonal Fisher information approximation (PerTA-fisher). Experiments show that PerTA consistently outperforms standard task vectors in forgetting effectiveness and overall model utility, and in many cases also exceeds commonly used training-based unlearning methods, while retaining the efficiency of task arithmetic.