The researchers propose the Rewrite to Jailbreak method, achieving transferable black-box jailbreaks by rewriting instructions.

Simply rewriting harmful instructions that are refused can make aligned LLMs generate content that violates usage policies; the authors state that such rewriting is learnable and can be transferred to other models and datasets.

• Scenario: The authors state that safety alignment causes LLMs to refuse non-compliant content, but jailbreaking can still turn originally refused instructions into instructions that elicit harmful outputs. The black-box setting does not access parameters or gradients, which the authors state is more realistic for actual jailbreak scenarios.
• Existing shortcomings: Existing approaches construct stricter instruction-following scenarios or search for adversarial prefixes or suffixes. The authors state their shortcomings are high probing costs; meaningless tokens or irrelevant scenarios are easily detected; prompts are targeted at specific goals and cannot be guaranteed to work for other cases and other LLMs.
• Observation: Starting from the idea that "when a model wrongly refuses, people rephrase," the authors believe that merely changing wording without adding extra features can jailbreak, and that rewriting strategies can be learned automatically.
• Threat model:
  • Goal: Keep the harmful intent unchanged and make aligned LLMs generate content that violates the service provider's usage policy.
  • Knowledge: The authors call it black-box, without access to target parameters and gradients.
  • Capability: The attack model rewrites user instructions; it can obtain target responses and harmfulness and similarity scores. For transfer, it is first trained on gθ, then used on gϕ without further fine-tuning.
  • Victim system: Aligned open-source or closed-source LLMs.
• Proposal: R2J (Rewrite to Jailbreak). The attack model iteratively explores weaknesses and updates the rewriting strategy. The authors state these rewrites can also be directly used for supervised fine-tuning to strengthen defenses.

The paper categorizes existing jailbreaks into forcing instruction adherence and adversarial prefixes/suffixes, and argues that neither is suitable for large-scale black-box attacks that are hard to detect.

Forcing instruction adherence

• Manual scenarios: Perez and Ribeiro (2022) and Mozes et al. (2023) design prompts that make instructions more strictly followed. The authors state that safety alignment may be weaker in such cases. Bhardwaj and Poria (2023) use role-playing for identity transformation; Qiu et al. (2023) disguise as translation; Ding et al. (2023) disguise as code writing.
• Automatic search: Ferret (Pala et al., 2024) searches for personas. TAP (Mehrotra et al., 2023), PAIR (related work cites Chao et al., 2023), and Puzzler (Chang et al., 2024) search scenarios based on meta-templates. Appendix A also discusses WordGame (Zhang et al., 2024) and Wolf in Sheep (Ding et al., 2023).
• Paper's comparison: Appendix A states that these methods create stronger adherence scenarios with conspicuous features; R2J does not introduce such scenarios, only restates the original intent. Table 7 provides comparison examples, but the rewritten sentences are not reproduced here.

Adversarial prefixes and suffixes

• GCG: Zou et al. (2023) use gradient search for suffixes to increase the probability of harmful responses. The authors state that it requires weights and cannot directly attack closed-source models like GPT and Gemini.
• AutoDAN: Zhu et al. (2023) use a hierarchical genetic algorithm to search for adversarial targets, also requiring weights.
• Detectability: The authors state that prefixes/suffixes may be unrelated to the instruction (Alon and Kamfonas, 2023), and after concatenation they are easy to detect. The introduction cites Hu et al. (2023) and Kumar et al. (2023), stating that checking for strange tokens and scenarios can distinguish attacks.

Baselines and datasets

• Comparison methods: DAN (King, 2023), CoT (Wei et al., 2022), CoT-2 (Wei et al., 2024), RedEval (Bhardwaj and Poria, 2023), TAP, PAIR (Table 1 writes Wei et al., 2022), ArtPrompt (Jiang et al., 2024), AutoDAN, AmpleGCG-plus (Table 1 as Kumar et al., 2024; one place in the text writes Zhu et al., 2023).
• Data: AdvBench (Chen et al., 2022), CatQA-en (Bhardwaj et al., 2024), HEx-PHI (Qi et al., 2024), DangerousQA (Bhardwaj and Poria, 2023). Without expected harmful targets, the authors state that PAIR, TAP, and AutoDAN are not applicable.

The authors position R2J as a black-box rewriting method: it does not rely on manual templates, does not add meaningless prefixes/suffixes, does not introduce irrelevant scenarios, and uses iterative learning to transfer rewriting to other datasets and models. The introduction also cites Sun et al. (2023) on safety alignment, Dong et al. (2024) on black-box attacks and defenses, and Liu et al. (2023) on the difficulty of transferring prompts for specific targets.

R2J uses an attack LLM to learn how to rewrite refused instructions, then filters based on the harmfulness and similarity of the target response, iteratively updating the attack model; the trained model can be transferred to other models or red-team sets without further fine-tuning.

Red-team data and attempt records

• Data objects: The k-th round red-team set is denoted Dθ^(k), containing N samples. Each Xi contains the original instruction Ii and several attempts; the j-th attempt Aji includes the rewrite Cij, target response Rij, and score Sij. The rewrite must express the same attack goal as Ii. Notation is given in Appendix B.
• Iteration source: The next round's red-team set is reconstructed from filtered rewrites. Before iteration begins, the authors use GPT-3.5-turbo to pre-generate the first round of rewrites; the paper does not specify whether this call is GPT-3.5-turbo-0125.
• Process: Figure 2 is divided into a training phase and a rewriting phase. In the training phase, the best-performing attempts are selected from existing attempts for SFT; in the rewriting phase, the current attack model rewrites these samples and re-scores them.

Training phase

• Filtering: Sort by evaluation score from high to low, and take the top p attempts per sample to form SFT data. The authors state that the same input can correspond to different desired rewrites, increasing diversity and avoiding overfitting to a particular rewrite.
• Objective: Equation (1) is f^(k)_gθ,Dθ=argmax_f∑_{(I,A)∈ D^(k)θ[:p]}∑_{C∈ A}log P_f(C|I). Meaning: on high-scoring attempts, maximize the log probability that the attack model generates rewrite C from original instruction I.
• Attack model: Implemented with an LLM, because the authors state rewrites are fluent and controllable via instructions. In experiments, the base is Llama-3-8b, fine-tuned with LoRA. Appendix E.4 gives the SFT sample format: input is the original instruction, output is the rewritten instruction.

Rewriting and scoring

• Rewriting: Same as training, first sort by score, take the top q attempts, construct inputs in the same format as SFT, generate new instructions with the current attack model, then pass them to the target model gθ to get responses.
• Scores: The evaluation model (the authors give GPT-3.5 as an example) scores both harmfulness and similarity, and the sum is used for ranking. Harmfulness is on a 1 to 5 scale, following Qi et al. (2024), combining whether it violates the service provider's usage policy and whether the response is relevant to the request. Similarity is also on a 1 to 5 scale, measuring whether the rewrite is still the same task. Appendix C provides the scoring template.
• Failure rule: For methods that generate instructions with an LLM (including PAIR, TAP, and R2J), if similarity is below 3, the attack is judged a failure, and the harmfulness in the result is recorded as 1. The authors state that GPT is more reluctant to give high scores, especially 5, so they set the similarity threshold at 3.

Transfer and implementation parameters

• Cross-model: After obtaining the attack model on open-source gθ, it is used for another model gϕ without further fine-tuning. The authors state that rewrites do not contain model-specific content, so they can be transferred to other red-team sets or models. Algorithm 2 rewrites only the current most effective attempt each round, querying the target once; it stops when harmfulness reaches 5. The query limit equals the number of iterations.
• Parameters: Training iterations n=20, p=5, q=3, with each selected attempt rewritten 3 times. Transfer n=30, with each sample queried at most 30 times (Appendix E.3). LoRA rank 16, learning rate 10^-4, total batch 32, DeepSpeed ZeRO-3 and AdamW; DangerousQA trained for 5 epochs, the other three datasets for 3 epochs.
• Sampling: All LLMs use the same default system prompt, with top_p of 1.0. Attack model temperature 1.0, target model 0.7, judge model 0. Appendix E.2 states SFT uses two NVIDIA A100s.

实验设置

• 受害模型：直接攻击为GPT-3.5-turbo-0125与Llama-2-7b-chat。Table 3的迁移目标为Llama-2-7b-chat、Llama-2-13b-chat、Llama-3-8b-chat、Gemini-pro、GPT-3.5-turbo-1106、GPT-3.5-turbo-0125、GPT-4-0125、GPT-4o-2024-05-13、GPT-4o-mini-2024-07。安全对齐另用Seungyoun/llama-2-7b-alpaca-gpt4。
• 攻击与评审：攻击模型为Llama-3-8b（LoRA）。有害性与相似度由GPT-3.5-turbo-0125打1到5分；该模型本身也是受害模型。LLM生成指令时，相似度低于3分判失败，有害性记1分。
• 数据：AdvBench 520条，含期望攻击输出；CatQA-en 550、HEx-PHI 330、DangerousQA 200，只有有害指令。首轮改写由GPT-3.5-turbo预生成。
• 基线：DAN、CoT、CoT-2、RedEval、TAP、PAIR、ArtPrompt、AutoDAN、AmpleGCG-plus。表中Initial（写作Inital）表示不做越狱。无期望有害目标时，作者称PAIR、TAP、AutoDAN不适用。
• 指标：Harmful为五分制有害性。ASR为关键词平均成功率；附录D列49个拒答信号，作者称该指标可能有假阳性与假阴性。迁移另报平均Queries，越低越好。
• 预算：训练n=20；迁移最多30次查询，有害性到5分停止。论文未报告攻击实验的重复次数。温度见第3问。

主结果

下表只列Table 1的一部分。AutoDAN在GPT-3.5-turbo-0125上为Not applicable，未填。

Swipe horizontally to read the table

模型	方法	Harmful	ASR	出处
GPT-3.5-turbo-0125	ArtPrompt	3.69	87.12%	Table 1
GPT-3.5-turbo-0125	AmpleGCG-plus	1.35	6.92%	Table 1
GPT-3.5-turbo-0125	R2J	4.20	86.54%	Table 1
Llama-2-7b-chat	PAIR	1.80	25.38%	Table 1
Llama-2-7b-chat	AutoDAN	3.06	52.69%	Table 1
Llama-2-7b-chat	R2J	3.79	75.38%	Table 1

• 作者的总括与表内反例：作者称R2J在两个模型上相对其他方法有超过20%的提升。Table 1中，GPT-3.5的ASR上ArtPrompt为87.12%，高于R2J的86.54%；R2J的Harmful 4.20为该列最高。Llama-2上R2J的Harmful与ASR均为表中最高。作者称相对AutoDAN提升22%，相对PAIR和TAP的ASR提升50%、Harmful约为2倍。
• 模板方法与换模型：作者称DAN、CoT、CoT-2目前在两个模型上都不能成功攻击。Table 1中GPT-3.5上三者ASR为0.00%、1.15%、2.31%；Llama-2上均为0.58%。RedEval在GPT-3.5上ASR为36.92%，在Llama-2上为0.19%。作者称ArtPrompt与AmpleGCG-plus换到另一模型时表现差，现有方法泛化有限；Llama-2上AmpleGCG-plus的Harmful为3.72、ASR为31.92%。
• 四个数据集：作者称两个模型在全部数据集上ASR超过80%。Table 2中Llama-2-7b-chat在AdvBench上为75.38%，不在这句话覆盖范围内。GPT-3.5上R2J的ASR另为CatQA-en 97.82%、DangerousQA 91.50%、HEx-PHI 88.18%；Llama-2上为93.27%、90.50%、82.73%。作者称四数据集每轮平均训练35,858 GFLOPS，两张A10上平均522秒。

跨模型迁移

• 协议：AdvBench上，用针对gθ训练的攻击模型攻击gϕ，不再微调。gθ为GPT-3.5时指GPT-3.5-turbo-0125，为Llama-2时指Llama-2-7b-chat。Table 3的“-”行是不做越狱。作者称共九个LLM，其中三个闭源、六个开源；具体名单以Table 3的gϕ列为准。
• 作者给出的汇总：作者称约20次查询下，针对GPT-3.5与Llama-2训练的攻击模型，在九个目标上的总Harmful为3.81与3.72，总ASR为79.77%与76.06%。该汇总含gϕ与gθ为同一模型的行。作者称相对直接训练、查询更少时只下降约8%。Table 1直接攻击GPT-3.5的ASR为86.54%；Table 3中gθ与gϕ均为GPT-3.5一侧时ASR为78.27%、查询14.87。Llama-2直接为75.38%，Table 3中gθ与gϕ均为Llama-2一侧时为68.27%、查询18.05。
• 不同目标：Table 3中，GPT-3.5所训模型打Llama-2-7b-chat的ASR为61.35%、查询22.08；打GPT-4o-mini-2024-07为89.62%、查询11.46；打Gemini-pro为83.08%、查询20.17。Llama-2所训模型打GPT-4o-2024-05-13的ASR为84.62%、查询12.49。作者认为迁移表现与目标能力可能相关，对更强目标更好。

跨数据集迁移

• Table 4：攻击模型在Dθ上训练后攻击AdvBench。GPT-3.5-turbo-0125上，来自CatQA-en、DangerousQA、HEx-PHI的ASR为73.27%、74.23%、56.15%；Llama-2-7b-chat上为63.85%、59.23%、42.88%。来自AdvBench的78.27%与68.27%，与Table 3中对应的同模型迁移行相同，不是Table 1的直接攻击数字。
• Figure 3：横轴为Number of Queries，纵轴分别为Harmful Score与Average Success Rate；图例含PAIR、TAP，以及四个红队集上的R2J。文本转换读不到曲线端点。作者称越靠近左上，成功率越高且查询越少；在Llama-2-7b-chat上，攻击效率是PAIR的2到3倍。
• 作者对弱项的解释：作者称HEx-PHI的有害类型与其他集不同，迁到GPT-3.5表现一般，但在相同查询数下，Llama-2上ASR比PAIR高20%。作者称DangerousQA不到AdvBench的40%，迁到AdvBench仍达到直接在AdvBench上训练时90%的攻击性能。Figure 3未在正文给出与PAIR逐点对齐的查询数。

防御、分布与安全对齐

• 扰动防御：Table 5在AdvBench上评对R2J的防御，论文未说明目标模型。w/o defense为Harmful 4.20、ASR 86.54%，与Table 1中GPT-3.5-turbo-0125的R2J一行相同。Paraphrasing为4.11、86.15%；Backtranslation为3.52、76.92%；RA-LLM为3.76、73.85%。作者称这些扰动只使Harmful与ASR下降17.20%与12.69%。
• 长度与PPL：Figure 4的目标是Llama-2-7b-chat、数据为AdvBench；横轴Perplexity，纵轴Length，均为文中所示的对数坐标。图例为w/o att.、AutoDAN、TAP、PAIR、R2J。文本读不到点的位置。作者称R2J与原指令分布重叠，因而PPL过滤和GBM不能防御；AutoDAN因无意义后缀提高PPL；PAIR与TAP因无关场景变长，与原指令有边界。正文提到compression（Liu et al., 2024b），Table 5没有对应数字。
• 安全对齐：Table 6在Seungyoun/llama-2-7b-alpaca-gpt4上用LoRA微调3个epoch。AdvBench-R2J对齐后，三个集平均Harmful 1.36、ASR 12.11%；原始AdvBench对齐为1.53、18.37%；不做SFT为3.49、60.16%。作者称相对原始AdvBench对齐，Harmful与ASR平均下降11%与6%。

其他消融与分析

• 附录F：作者称改写后平均词长从5.3增至6.1个字符；直接表达攻击意图的词减少，委婉用词增加。此处不列具体替换词。
• 附录C.3：2500条（每个分数500条）、10轮重评，Harmful与相似度方差均为0.0001；均值2.9758与2.9867，期望均为3.0000。
• 人工：50条、5名专家，Cohen's Kappa为0.773（有害性）与0.698（相似度）；以3分为界时为0.913与0.801。
• Figure 5：作者称第4.2节各模型迁移攻击的平均相似度都高于3.8。
• 硬件表述：附录E.2写SFT用两张A100；第4.1节写计时用两张A10、平均522秒。论文未把两处对应起来。
• 论文未报告对p、q或迭代轮数的对比实验。附录G给了生成示例，含未过滤有害内容，此处不复述。

Limitations and future directions noted by the authors

• Attack base model: Other base models as attack models were not explored. The authors state that in theory R2J does not depend on the base model, and closed-source models could even be fine-tuned via API. (Limitations)
• Language: Attack effectiveness in other languages was not provided. The authors state that real-world jailbreaks are not limited to one language, and in theory rewriting attacks could be done on any language, but no experiments were conducted. (Limitations)
• Why not done: The authors state that the paper focuses on attack performance and transferability, so the above two points were not discussed. (Limitations)

Experimental coverage

• Red-teaming data are AdvBench (520), CatQA-en (550), HEx-PHI (330), DangerousQA (200); the Limitations state that these four are English red-teaming sets. (Section 4, Limitations)
• Victim models for direct attacks are GPT-3.5-turbo-0125 and Llama-2-7b-chat; the attack model is Llama-3-8b, fine-tuned with LoRA. (Section 4, Appendix E)
• Transfer targets in Table 3 are Llama-2-7b-chat, Llama-2-13b-chat, Llama-3-8b-chat, Gemini-pro, GPT-3.5-turbo-1106, GPT-3.5-turbo-0125, GPT-4-0125, GPT-4o-2024-05-13, GPT-4o-mini-2024-07. The authors state that three of these are closed-source and six are open-source.
• Defenses with numbers are paraphrasing, backtranslation, RA-LLM (Table 5, target model not specified). The safety-aligned model is Seungyoun/llama-2-7b-alpaca-gpt4, evaluated on CatQA-en, DangerousQA, HEx-PHI (Table 6). AutoDAN is marked Not applicable on GPT-3.5, and the authors state that this method requires weights.
• The judge model is GPT-3.5-turbo-0125; the human agreement sample is 50 items with 5 experts (Appendix C.3). Training iterations n=20, with at most 30 queries per item for transfer (Appendix E.3). The paper does not report the number of repetitions for attack experiments.

R2J is a transferable black-box jailbreak: an attack model rewrites rejected harmful instructions into statements with the same intent.

• Problem: The authors state that forcing adherence to scenarios and adversarial prefixes/suffixes have high probing costs, are conspicuous, and do not guarantee transfer to other targets and models. The black-box setting does not access parameters or gradients.
• Method: Filter rewrites by the sum of harmfulness and similarity, and apply LoRA to Llama-3-8b. The scorer is GPT-3.5-turbo-0125; similarity below 3 is recorded as failure. When transferring to new models or datasets, no further fine-tuning is done, with at most 30 queries per instruction.
• Direct attack: On AdvBench, GPT-3.5-turbo-0125 achieves Harmful/ASR of 4.20/86.54%, and Llama-2-7b-chat achieves 3.79/75.38% (Table 1). In the same table, ArtPrompt has an ASR of 87.12% on GPT-3.5. On CatQA-en, GPT-3.5 achieves an ASR of 97.82% (Table 2). The authors state that all four datasets exceed 80%, but in Table 2, Llama-2's AdvBench is 75.38%.
• Transfer: The authors state that attack models trained on GPT-3.5 and Llama-2 achieve total ASRs of 79.77% and 76.06% across nine targets, with about 20 queries, roughly 8% lower than direct training. In Table 3, the model trained on GPT-3.5 achieves an ASR of 89.62% against GPT-4o-mini and 61.35% against Llama-2-7b-chat.
• Authors' conclusion: The authors state that experiments demonstrate R2J's effectiveness and cross-model, cross-dataset transferability, without introducing meaningless prefixes/suffixes or irrelevant scenarios. In Table 6, after safety alignment using data generated by R2J, the average ASR across three datasets is 12.11%, compared to 18.37% when aligning with the original AdvBench. The authors state that R2J can also be used to help build safer LLMs.