Meta AI and CMU researchers reduce GCG jailbreak attack success rate by up to 22% using pre-deployment training data filtering
Meta AI and CMU Researchers Cut AI Jailbreak Rates by 22% With Pre-Deployment Training Data Filter
Researchers from CMU, the University of Washington, Meta AI and other institutions published a paper on arXiv this week proposing a gradient attribution method requiring no safety annotations, SimpleLoRIF, and a filtered dataset, SafeTulu, which reduce the GCG jailbreak attack success rate by up to 22% across the Qwen, Llama, and OLMo model families, while GSM8K and MATH 500 mathematical reasoning performance is on par or better. The method compares training samples with unsafe outputs by cosine similarity in gradient space, removes the most relevant 20% of samples and retrains from the base model, and backpropagates only through the last four Transformer blocks to control compute. Under HarmBench transfer GCG attacks, the SafeTulu models have attack success rates of 3.1% and 3.8%, lower than full Tulu 3's 9.4% and 10.2%, while LLM-as-a-judge filtering of the same proportion of samples shows almost no improvement, indicating that the safety degradation signal is not at the text level.
- Following the Gradient: Tracing Large Model Safety Degradation to Training Data
- Understanding and Preserving the Safety of Fine-Tuned Large Language Models
- No, of course I can: deeper fine-tuning attacks that bypass token-level safety mechanisms
- Universal and transferable adversarial attacks against aligned language models