Understanding and Preserving the Safety of Fine-Tuned Large Language Models

The paper proposes a safety-preserving fine-tuning method, SPF (safety-preserving fine-tuning), to address the problem of fine-tuning damaging the safety alignment of large language models. Through systematic empirical analysis, the authors present three findings: safety gradients lie in a low-rank subspace while utility gradients span a higher-dimensional space; the two subspaces are often negatively correlated and produce directional conflicts during fine-tuning; and the dominant safety direction can be efficiently estimated from a single sample. Based on these findings, SPF explicitly removes gradient components that conflict with the low-rank safety subspace, with theoretical guarantees of utility convergence and bounded safety drift. Experiments show that SPF preserves downstream task performance and restores almost all pre-trained safety alignment, remains effective in adversarial fine-tuning scenarios, and exhibits robust resistance to deep fine-tuning and dynamic jailbreak attacks.