CKU: Constrained Knowledge Unlearning for LLM Safety Alignment
Safety Alignment via Constrained Knowledge Unlearning
Translation in progress — untranslated text is omitted.
The researchers propose a safety alignment method called Constrained Knowledge Unlearning (CKU): first, score neurons in specific MLP layers to identify a subset U of neurons related to useful knowledge, then clip the gradients of these neurons during the unlearning process to suppress harmful content while preserving useful knowledge. Experiments show that CKU improves model safety without compromising overall performance, and its sensitivity analysis of MLP layer neurons also provides a reference for safety alignment and knowledge editing mechanisms.
- Aligned LLMs can still be induced by jailbreaks to produce harmful outputs. The authors believe continued training does not delete harmful knowledge and may also reduce general capability or learn irrelevant knowledge.
- CKU uses SNIP to score MLP neurons, clips the gradients of neurons related to useful knowledge, and then performs gradient ascent on harmful answers with a λ constraint. Figure 2 fixes the top 80% of neurons by score.
- The authors compare five baselines and four jailbreaks on Llama2-7B-Chat and Llama3-8B-Instruct. The authors state CKU has lower ASR in most cases and average utility close to the original model; in Table 1, GCG's AdvBench ASR is lower for RSFT.
- Limitations states: effectiveness varies with domain or dataset, occasionally difficult to give nuanced explanations in complex or ambiguous contexts, and generality and efficiency still need improvement. Experiments cover two aligned models, four attacks, five baselines; some test set sample sizes and repetition counts are not reported.
Related papers
Both use: RTE
Both use: CommonsenseQA
Show 3 more papers
Both use: CommonsenseQA
Both use: MaliciousInstruct