JWP: Joint Weight Perturbation Defends Against Harmful Fine-Tuning
Start here
The paper proposes the Joint Weight Perturbation (JWP) defense method: the authors observe that when the disjoint parts of a harmful perturbation are applied individually their effect is limited, but when applied jointly they make the model's harmfulness exceed the sum of the effects of the individual parts. JWP exposes the model to different combinations of harmful perturbations during the safety alignment stage, training its safe responses to reduce sensitivity to the cumulative effects of perturbations in subsequent harmful fine-tuning. Experiments on downstream tasks and across multiple model series show that JWP's resistance to harmful fine-tuning is stronger than that of prior defense methods, while largely preserving task performance.
Generated by AI from reports · Updated 24 days ago
Attention and developments
Attention, hourly
Now 27, 0% in 24 hours·Peak 27 (Nov 6 03:00)
Related research
This event's paper
- When Safety Fails Collectively: Robust Alignment Against Harmful Fine-Tuning via Joint Weight Perturbations
arXiv · 2026-10-19 · Deep reading
Progress timeline
- XiaohongshuXiaohongshu · Liu Dongrui (Shanghai AI Lab)· DiscussionJoint weight perturbation defends against malicious fine-tuning of LLMs(opens the original post in a new tab)
- arXiv: Jailbreaks, Prompt Injection, Poisoning & DefensesWhen Safety Fails Collectively: Robust Alignment Against Harmful Fine-Tuning via Joint Weight Perturbation
The paper proposes joint weight perturbation (JWP), a defense method that, during the safety alignment stage, masks different parts of harmful perturbations and trains the model to respond safely, in order to improve large language models' resistance to subsequent harmful fine-tuning. The authors observe a collective failure mode: when the disjoint parts of a harmful perturbation are applied individually, their effect is limited, but when applied jointly, they make the model more harmful than the sum of the effects of the individual parts. JWP exposes the model to different combinations of harmful perturbations, reducing its sensitivity to the cumulative effects of perturbations during subsequent harmful fine-tuning. Experiments on downstream tasks and across multiple model series show that JWP's resistance to harmful fine-tuning is stronger than that of prior defense methods, while largely preserving task performance.
Related discussion
- Older than 72 hours · Not included in the 72-hour figures
- XiaohongshuXiaohongshu · Liu Dongrui (Shanghai AI Lab)Joint weight perturbation defends against malicious fine-tuning of LLMs(opens the original post in a new tab)