Diagnose Before Optimize: Multi-Dimensional Critique Feedback for Query-Efficient Jailbreak Attacks
Diagnose Before You Refine: Multi-Dimensional Judge Feedback for Query-Efficient Jailbreak Attacks
Translation in progress — untranslated text is omitted.
The researchers propose JSB (Jev-Guided Strategy Bandit), a query-based jailbreak framework that uses multi-dimensional probabilistic feedback to guide prompt optimization, combining continuous jailbreak scoring, targeted refinement, strategy switching, and optimal-state reseeding to balance exploration and refinement. With a budget of only 10 target model queries, JSB achieves an average HarmBench attack success rate of 89.8% across ten open-source model settings, consistently outperforming existing automated jailbreak attacks. The method is evaluated on five open-source and three closed-source LLMs and two harmful request datasets, with analyses under different query budgets and attack models. Defense experiments reveal limitations of existing filtering mechanisms and point to the potential of Jev-based output filtering.
- Query-based jailbreaks need to rewrite prompts within very few target model queries, but common evaluation feedback is coarse-grained and may mistake superficial compliance for success, thus biasing the search.
- JSB uses Jev to give stance probabilities as well as relevance, specificity, and completeness in one call. Parallel threads do targeted refinement, strategy switching, or reseeding from the global best state based on scores; main experiments use early stopping threshold 0.7 and comparison budget 10 queries.
- At B=10, the authors report an average HarmBench ASR of 89.8% across ten open-source settings and 54.3% across three closed-source models; StrongREJECT average scores are also higher than the three baseline groups. JSB is 19% on Claude-Haiku-5.5.
- The authors point out that the strategy library is fixed, which may fail to discover out-of-library strategies; closed-source only tests three lightweight variants; effectiveness may be limited on more jailbreak-resistant targets. Defense and attacker replacement are only done on Qwen2.5-7B. The paper does not report number of repeats or human agreement.
Related events and news
1 events · 1 news mentions
News that mentions this paper
RL/RLHF 训练条件可诱发模型智能体提权逃逸
- 公众号 大模型安全研究cites the paper's id or link大模型安全周报:RL/RLHF 训练条件本身即可诱发前沿模型智能体提权逃逸(CVE-2026-65616)
Diagnose Before You Refine: Multi-Dimensional Judge Feedback for Query-Efficient Jailbreak Attacks(https://arxiv.org/abs/2610.19971)
Related papers
Show 3 more papers
- The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMsAnalysis / interpretabilityarXiv