GradShield: Fine-Tuning While Preserving Alignment

The paper proposes GradShield, a filtering method that identifies and removes harmful data points before fine-tuning, used to reduce safety alignment degradation after model fine-tuning. The method computes a Fine-tuning Implicit Harmfulness Score (FIHS) for each data point and uses an adaptive threshold algorithm to filter data. The authors evaluate on multiple utility fine-tuning tasks and at different harmful data levels, and report that GradShield outperforms all baseline methods, keeping attack success rate below 6% while maintaining utility performance.