ATTCK-R: A Knowledge-Aware Long-Horizon Jailbreak Framework for Protected LLMs

AI summaryThe researchers propose a multi-turn red-teaming framework ATTCK-R for conducting jailbreak tests on black-box systems equipped with prompt-based safety defenses. The paper reports that after introducing a layer of prompt-based safety defense, the average attack success rate of a strong multi-turn baseline drops from 0.896 to 0.608, a relative decrease of 32.1%, indicating that evaluations without defenses significantly overestimate vulnerability in real deployments. ATTCK-R models jailbreaking as a long-horizon trajectory optimization problem, uses structured belief states, distills reusable strategies from past successes and failures through optimization-based strategy screening, and uses informed backtracking to continue attacks from strategically meaningful states rather than restarting. Across 9 common LLMs configured with a single layer of prompt-based safety defense, ATTCK-R consistently outperforms recent strong multi-turn jailbreak frameworks. Ablation results show that long-horizon planning and strategy screening improve attack effectiveness, while adaptive backtracking improves efficiency by structurally recovering from conversation failures.