CRAFT: Contrastive reasoning alignment via reinforcement learning from hidden representations
Contrastive Reasoning Alignment: Reinforcement Learning from Hidden Representations
AI summary
The red-teaming alignment framework CRAFT improves the reasoning safety of Qwen3-4B-Thinking and R1-Distill-Llama-8B against jailbreak attacks by optimizing contrastive representations in the hidden state space and reinforcement learning objectives. On multiple safety benchmarks, its reasoning safety and final response safety improve by an average of 79.0% and 87.7% over the base models, outperforming IPO and SafeKey. The method also incorporates latent–text consistency into GRPO, ruling out surface safety strategies as a local optimum.