Skip to content

Diagnose Before Optimize: Multi-Dimensional Critique Feedback for Query-Efficient Jailbreak Attacks

Diagnose Before You Refine: Multi-Dimensional Judge Feedback for Query-Efficient Jailbreak Attacks

arXiv: Adversarial Attacks, Backdoors & Poisoning· arXiv:2610.19971· Yixin Tan, Chia-Yi Hsu, Jiayang Liu, Chia-Mu Yu, Rui WenInstitute of Science Tokyo; National Yang Ming Chiao Tung University; Nanyang Technological UniversityIndexed by this site · 2026-10-15· Originally publishedAttacks

Translation in progress — untranslated text is omitted.

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

AI summaryThe researchers propose JSB (Jev-Guided Strategy Bandit), a query-based jailbreak framework that uses multi-dimensional probabilistic feedback to guide prompt optimization, combining continuous jailbreak scoring, targeted refinement, strategy switching, and optimal-state reseeding to balance exploration and refinement. With a budget of only 10 target model queries, JSB achieves an average HarmBench attack success rate of 89.8% across ten open-source model settings, consistently outperforming existing automated jailbreak attacks. The method is evaluated on five open-source and three closed-source LLMs and two harmful request datasets, with analyses under different query budgets and attack models. Defense experiments reveal limitations of existing filtering mechanisms and point to the potential of Jev-based output filtering.

Questions
Query-based jailbreaks need to rewrite prompts within very few target model queries, but common evaluation feedback is coarse-grained and may mistake superficial compliance for success, thus biasing the search.
Methods
JSB uses Jev to give stance probabilities as well as relevance, specificity, and completeness in one call. Parallel threads do targeted refinement, strategy switching, or reseeding from the global best state based on scores; main experiments use early stopping threshold 0.7 and comparison budget 10 queries.
Experiments and results
At B=10, the authors report an average HarmBench ASR of 89.8% across ten open-source settings and 54.3% across three closed-source models; StrongREJECT average scores are also higher than the three baseline groups. JSB is 19% on Claude-Haiku-5.5.
Limitations and future work
The authors point out that the strategy library is fixed, which may fail to discover out-of-library strategies; closed-source only tests three lightweight variants; effectiveness may be limited on more jailbreak-resistant targets. Defense and attacker replacement are only done on Qwen2.5-7B. The paper does not report number of repeats or human agreement.

Related events and news

1 events · 1 news mentions

News that mentions this paper

RL/RLHF 训练条件可诱发模型智能体提权逃逸

Related papers

Show 3 more papers
Read originalarxiv.org