Meta AI, CMU, and others propose SimpleLoRIF to filter training data and reduce the jailbreak rate by 22%
Start here
Researchers from CMU, the University of Washington, Meta AI and other institutions published a paper on arXiv this week proposing a gradient attribution method that requires no safety labels, SimpleLoRIF, and a filtered dataset, SafeTulu. The method compares the cosine similarity between training samples and unsafe outputs in gradient space, removes the 20% most relevant samples, retrains from the base model, and adjusts only the last four Transformer layers. Across the Qwen, Llama and OLMo model families, the study reduced the success rate of GCG jailbreak attacks by up to 22%, while mathematical reasoning performance on GSM8K and MATH 500 was on par or better.
Generated by AI from reports · Updated 13 days ago
Progress timeline
- TechTimesMeta AI and CMU researchers reduce GCG jailbreak attack success rate by up to 22% using pre-deployment training data filtering
Researchers from CMU, the University of Washington, Meta AI and other institutions published a paper on arXiv this week proposing a gradient attribution method requiring no safety annotations, SimpleLoRIF, and a filtered dataset, SafeTulu, which reduce the GCG jailbreak attack success rate by up to 22% across the Qwen, Llama, and OLMo model families, while GSM8K and MATH 500 mathematical reasoning performance is on par or better. The method compares training samples with unsafe outputs by cosine similarity in gradient space, removes the most relevant 20% of samples and retrains from the base model, and backpropagates only through the last four Transformer blocks to control compute. Under HarmBench transfer GCG attacks, the SafeTulu models have attack success rates of 3.1% and 3.8%, lower than full Tulu 3's 9.4% and 10.2%, while LLM-as-a-judge filtering of the same proportion of samples shows almost no improvement, indicating that the safety degradation signal is not at the text level.