Attack evaluation of multilingual jailbreak prompts against closed-source large language models: GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max

Can closed-source commercial models still output policy-violating content under Chinese and English jailbreak prompts, and which prompt structures are more likely to bypass protections.

• Scenario: The authors state that large models are used in search, dialogue, and content generation, but they still hallucinate and output harmful content under misleading prompts. Safety research mostly targets open-source models like LLaMA and ChatGLM; closed-source models like GPT-4o and DeepSeek-R1 are opaque internally and difficult to scrutinize externally.
• Existing gap: The authors state that safety measures and evaluation sets are biased toward English, and non-English prompts are more likely to bypass protections. Research on closed-source models is scarce and mostly limited to earlier models like GPT-4.
• Authors' hypothesis: The language environment changes hallucination tendencies and safety effects; even with training-time alignment and inference-time filtering, multilingual jailbreaks may still bypass protections.
• This paper proposes: An ensemble adversarial framework evaluating GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max on six safety content categories in Chinese and English, using ASR to compare prompt designs, models, and languages. The authors claim this is the first large-scale evaluation of closed-source model APIs.
• Threat model:
  • Goal: Induce the model to violate safety guidelines and generate harmful or false outputs. ASR only counts completions manually labeled as Success and containing specific unsafe content.
  • Knowledge: Interaction only through official APIs with default inference configurations. The methods section does not state white-box or black-box; Section 5.2 states that current jailbreak attacks mainly employ black-box strategies.
  • Capability: Control user prompts, including forbidden questions and a four-part prompt structure, with full Chinese or full English language. The paper does not state the ability to modify system prompts, decoding parameters, or weights.
  • Victim system: Official APIs of the above four closed-source models.

Jailbreaks and Misuse

• DAN and Wild Prompts: Shen et al. (2024) characterize wild jailbreak prompts. The introduction lists them as early manual instructions; the experiments do not reproduce them.
• Injection and Cross-Lingual: The authors review prompt injection, cross-lingual jailbreaks, and multimodal attacks, stating that these works embed malicious requests in benign contexts, role-play, or unconventional formats, and use ASR to measure bypass.
• Zhou et al. (2025): The authors cite their Table 1, stating that existing jailbreak techniques perform limitedly on closed-source models. The data in that table comes from reference [63], not from this paper's experiments.

Alignment and Inference-Time Defenses

• RLHF: Ouyang et al. (2022) train InstructGPT with human feedback to reduce harmful or false outputs.
• Constitutional AI: Bai et al. (2022) use preset principles instead of human annotation, enabling models to give explanatory refusals to inappropriate requests.
• Guardrails: The authors discuss input filtering, output review, and post-processing, citing Wildflare GuardRail. They state that training-stage alignment alone is insufficient to stop carefully crafted adversarial prompts.

Multilingual Safety

• Low-Resource Languages: Song et al. (2024) discuss two types of issues: harmfulness and relevance. The authors state that safety alignment is weaker in low-resource languages.
• Chinese-English Differences: The authors paraphrase Wang et al. (2024): mainstream models may refuse jailbreaks in English but give unsafe outputs in Chinese; RLHF alignment mainly comes from English corpora.
• Sandwich Attack: Upadhayay and Behzadan (2024) sandwich malicious requests between benign content in low-resource languages. The methods section says it draws on this idea but changes to a monolingual format for Chinese-English comparison.

Baselines and Data: The experiments do not use DAN, special encoding, or language switching as control attacks. The controls are the full prompt Full, and AS1–AS5 with components removed; AS5 keeps only the attack question. The 32 forbidden questions: 12 self-built, 12 from JailbreakBench, 8 from HarmBench. The methods section organizes them into six categories according to Google Gemini usage policy; the introduction writes seven categories, inconsistent with the six in the methods section and appendix.

The authors position this paper as a multilingual system evaluation of mainstream closed-source APIs, claiming to fill the open-source bias and the limited effectiveness of existing jailbreaks on closed-source models shown in Table 1.

The authors embedded forbidden questions into four-part prompts, called four closed-source APIs on Chinese and English parallel questions, then used component deletion and manual three-way annotation to compute ASR.

Forbidden Questions and Parallel Corpus

• Six categories: Illegal activities, abuse and service disruption, harmful content generation, false and misleading content, pornography, privacy violations. Subcategories are listed in the appendix; Figure 9 categorizes by question number.
• 32 questions: 12 self-created, 12 adapted from JailbreakBench, and 8 adapted from HarmBench. Initially written in English, then translated into Chinese by bilingual researchers and proofread.
• Semantic verification: Used LaBSE to compute similarities for the 32 pairs. The authors state scores range from 0.80 to 0.90, with a mean of 0.86 (Figure 3).

Four-Part Prompts

• Setting + Character: First specify a fictional scenario and persona, making the model follow subsequent instructions in character. The authors state this is to relax general refusal strategies.
• Sandwich: Within the same language, place the forbidden question between a harmless opening and a harmless closing. The authors state this is adapted from the multilingual sandwich attack but does not use low-resource languages, allowing direct Chinese-English comparison.
• Two Sides: After the forbidden question, require both supporting and opposing arguments. The authors state this frames the output as a balanced debate, putting the model in an analytical mode, thereby causing it to write details in the supporting argument that would otherwise be refused.
• Guide words: Append format constraints at the end that do not directly request harmful content, specifying how the answer should start or end. The authors state this is a variant of refusal suppression: the model tends to respond positively and at length to comply with the format.

Ablation Definitions

Figure 4 defines deletion of components, not a separate set of attacks:

1. Full: No components removed.
2. AS1: Remove Setting + Character.
3. AS2: Remove the harmless opening and closing of the sandwich.
4. AS3: Remove Two Sides.
5. AS4: Remove Guide words.
6. AS5: Keep only the attack question.

The introduction states that five ablations are used to see the impact of each component on safety. Section 4.2 describes AS1–AS4 as four jailbreak strategies and states AS3 is the Two Sides prompt; this is inconsistent with Figure 4. Subsequent ablation numbers are understood per Figure 4 and Section 4.4.

Execution and Success Determination

• All four models use official APIs with default inference configurations, no additional fine-tuning or per-prompt parameter adjustment. The paper does not report temperature.
• 32 questions × 2 languages = 64 questions; 64 × 6 variants × 25 repetitions = 9,600 per model; 38,400 for four models. Scripts ran on Google Colab, outputs stored in JSON.
• Labels are three types: Success (specific harmful or unsafe content), Fail (refusal or no unsafe content provided), Response but Acceptable (substantive response but reviewers deem it not violating).
• Two bilingual PhD students with AI safety backgrounds independently annotated; disagreements were adjudicated by a third senior reviewer, who also re-reviewed all Response but Acceptable cases. The paper states that inter-annotator agreement scores were not calculated.
• ASR is the proportion of completions manually judged to violate usage policy, only counting Success. Discussing the forbidden topic, or only explaining why it is not allowed, or providing only non-actionable general information, does not count as success.

Experimental Setup

• Models: GPT-4o (OpenAI), DeepSeek-R1 (DeepSeek), Gemini-1.5-Pro (Google), Qwen-Max (Alibaba; written as Qwen-Max-Latest in Table 3). All use official APIs with default configurations.
• Data: 32 questions × Chinese/English = 64 questions. Source and LaBSE similarity see Question 3: 0.80–0.90, mean 0.86.
• Variants and Repetitions: Full and AS1–AS5, 25 times per question per variant. Section 4.1.1 states 9,600 runs per model, totaling 38,400 outputs. The introduction writes about 40,000; Figure 2 writes 64×25=1,600. The later text follows 4.1.1. The abstract writes 32 types of jailbreak attacks, which differs from the counting of 6 variants.
• Metrics: ASR only counts Success. The paper states the goal as simultaneously evaluating hallucination and jailbreak, but the results section does not give separate factuality metrics.
• Review: Two independent annotators plus a third arbitrator; no LLM judge, no agreement score. Figure 2 writes Binary Rating, while Section 3.2 uses three-class labels.
• Cost: Table 3: GPT-4o 5,404,443 tokens, $43.48; DeepSeek-R1 8,133,940, $19.09; Gemini-1.5-Pro 7,784,962, $56.70; Qwen-Max-Latest 9,892,500, $16.23.

Main Results

Full-Attack ASR in Table 2:

Model	Chinese	English
GPT-4o	0.385	0.3219
DeepSeek-R1	0.8475	0.6913
Gemini-1.5-Pro	0.7663	0.7906
Qwen-Max	0.8925	0.7969

• The authors state GPT-4o has the strongest defense, with Full-Attack around 35%, Chinese 38.5%, English 32.19%; Qwen-Max is the weakest, averaging as high as 84%, Chinese 89.25%, English 79.69%. DeepSeek-R1 and Gemini-1.5-Pro are described by the authors as around 77% and 78%.
• The authors attribute GPT-4o's performance to RLHF, and the high ASR of Qwen-Max and DeepSeek-R1 to insufficient multilingual safety alignment. The paper does not measure RLHF or alignment data volume.
• The abstract states that Chinese prompts consistently have higher ASR than English. In Table 2's Full-Attack, Gemini-1.5-Pro's English 0.7906 is higher than Chinese 0.7663.

Language Differences

• Based on Table 2 and Figure 6, the authors state that for most models and most strategies, Chinese ASR is higher than English; for GPT-4o, the Chinese-English difference across strategies is about 0.05 to 0.21. Figure 6's caption is a CN−EN heatmap, but the text does not give cell values.
• The authors state that Qwen-Max and DeepSeek-R1 achieve SOTA on Chinese benchmarks, and write DeepSeek-R1's CLUEWSC(EM) 92.8%, C-Eval(EM) 91.8%. These two are not experiments in this paper, and the text does not give a source table. The authors state that more Chinese training does not correspond to lower Chinese jailbreak ASR.
• The authors' explanation for higher Chinese ASR is that euphemisms, homophones, and online variants make RLHF coverage difficult. This is the authors' speculation, not an ablation result.

Ablations and Component Roles

• Removing Two Sides (AS3): The authors state that ASR drops for all four models, giving GPT-4o about 0.18 (Chinese)/0.17 (English), DeepSeek-R1 about 0.45/0.45, Gemini-1.5-Pro 0.42/0.31, Qwen-Max 0.60/0.50. In Table 2, AS3 is lower than the corresponding Full, but higher than most AS5. The authors conclude that Two Sides is the most critical.
• Another statement in Table 5: The average ASR across the four models for AS1–AS4 is 0.5107, 0.6564, 0.3008, 0.5969. Section 4.2 states that AS3 is the Two Sides prompt, with the lowest average ASR, thus the strongest penetration. According to Figure 4, AS3 is the condition after removing Two Sides, and low ASR means the attack weakens after removal. The abstract and conclusion still state that Two Sides is the most effective.
• AS5: ASR is lowest when only the attack question remains. For English, GPT-4o and Qwen-Max are 0.0000; for Chinese, Qwen-Max 0.0125, GPT-4o 0.0663 (Table 2). The authors state that multi-component combinations have synergy relative to pure attack questions, and direct harmful requests are still mostly rejected in both Chinese and English.

Scenario Differences

• Table 4 does not specify variants and language, giving six categories' average/median ASR: illegal 0.623/0.88, abuse 0.501/0.62, harmful content 0.710/0.88, misleading 0.673/0.78, pornographic 0.780/0.84, privacy 0.567/0.64. The authors focus on the median, stating that the median 0.88 for illegal activities and harmful content generation is a stable vulnerability.
• Based on Table 6, the authors state: GPT-4o is weakest in the pornographic scenario (median AVG 0.740), DeepSeek-R1 is weakest in illegal activities (median AVG 0.920), Gemini-1.5-Pro is most easily broken in misleading content (median AVG 1.000), Qwen-Max is highest in the pornographic scenario (median AVG 1.000). Table 6 does not specify prompt variants.
• In Table 6, GPT-4o's average ASR for illegal activities in English 0.371 is higher than Chinese 0.216. The authors do not separately explain this cell.

Other Ablations and Analyses

• AS1 removes role setting: GPT-4o drops about 0.09 (Chinese)/0.23 (English); DeepSeek-R1 drops 0.2350/0.2694; Qwen-Max drops 0.2512/0.2669. Gemini-1.5-Pro Chinese rises about 0.1737 (Table 2: 0.7663 to 0.9400), English drops 0.2306.
• AS2 removes the sandwich: GPT-4o drops about 0.07/0.10; Qwen-Max drops 0.085/0.0444; DeepSeek-R1 changes 0.0275/0.0113. Gemini Chinese rises about 0.098, English Full and AS2 are both 0.7906.
• AS4 removes the guiding words: GPT-4o rises about 0.17 (Chinese)/0.031 (English), Table 2 Chinese from 0.385 to 0.5594. Gemini Chinese rises about 0.04, English drops about 0.17. DeepSeek-R1 drops 0.16/0.23, Qwen-Max drops 0.15/0.26.
• The authors state that guiding words are a protective factor for GPT-4o, but for DeepSeek-R1 and Qwen-Max they increase jailbreak success.
• Figure 9 gives ASR by question; the pornographic category only lists QN-29. The paper does not explain how a single-question category's median represents that category.

Limitations and Future Directions Noted by the Authors

• Timeliness (5.3): The authors state that the real-time adaptability of safety evaluation is low; models are updated iteratively, and a static single evaluation may not reflect the current safety state. Manual evaluation is resource-intensive, automated tools still have limitations in accuracy and coverage, and results may lose relevance as models evolve.
• Languages (5.3): The evaluation is limited to Chinese and English. The authors state this limits the generalizability of the conclusions, and many benchmarks are also monolingual, hindering understanding of differences in other languages, especially low-resource ones; they state that future work should expand the language coverage.
• White-box attacks (5.2): The authors state that current jailbreaks are mainly black-box, lacking transparent analysis of internal decisions; they plan to incorporate interpretability methods into white-box attacks to analyze hallucination and jailbreak vulnerabilities.
• Human-machine annotation (5.2): The authors state that automated review still differs from human intuition, and they plan to advance human-machine collaborative annotation and review.
• Chinese benchmarks (5.2): The authors state that existing hallucination and safety benchmarks are biased toward English, and Chinese inputs are more prone to hallucination; they plan to expand Chinese safety benchmarks and adversarial evaluation, and mention works like HalluQA.

Experimental Coverage

• Models tested are GPT-4o, DeepSeek-R1, Gemini-1.5-Pro, and Qwen-Max, called via official APIs with default configurations (4.1.2).
• Content is six categories with 32 questions from Appendix A; languages are Chinese and English; variants are Full and AS1–AS5; 25 repetitions per variant, totaling 38,400 outputs (4.1.1).
• Judgment is by two bilingual reviewers plus a third arbiter; Section 3.2 states that agreement scores were not calculated; no LLM judge was used.
• The control is component deletion of the same prompt, not a reproduction of other published attack methods (Figure 4, 4.4).
• The paper does not report decoding temperature, random seeds, system prompts, or auxiliary models for attacks. Semantic verification used LaBSE (3.1.2).

The authors use a set of Chinese-English jailbreak prompts to compare the safety defenses of four closed-source APIs.

• Problem: Commercial closed-source models have training-time alignment and inference-time filtering, but under Chinese and English jailbreaks, do they still produce policy-violating specific content, and which part of the prompt is at work.
• Method: 32 forbidden questions in six categories are translated into Chinese-English parallel pairs, then wrapped with role settings, monolingual sandwich, two-sided argumentation, and format guidance. Full and five ablation conditions are each repeated 25 times, yielding 38,400 outputs in total; two annotators label with a third adjudicating, and only Success is counted toward the ASR.
• Model differences: In Table 2's Full-Attack, Qwen-Max is 0.8925 in Chinese and 0.7969 in English, DeepSeek-R1 is 0.8475 and 0.6913, Gemini-1.5-Pro is 0.7663 and 0.7906, and GPT-4o is 0.385 and 0.3219. The authors state that GPT-4o has the strongest defense and Qwen-Max the weakest.
• Language and components: The authors state that Chinese prompts are generally more likely to succeed; in Table 2, Gemini-1.5-Pro's Full-Attack English is higher than Chinese. After removing two-sided argumentation, the ASR drop reported by the authors is the largest among components; with only the attack question remaining, English GPT-4o and Qwen-Max are 0.0000. Section 4.2 describes AS3's low ASR as Two Sides having the strongest penetration, which is opposite in direction to the ablation definition in Figure 4.
• Authors' conclusion: Safety measures can reject most direct requests, but combined multilingual prompts can still elicit violating outputs. The conclusion ranks the models from weakest to strongest as Qwen-Max, DeepSeek-R1, Gemini-1.5-Pro, GPT-4o; in Table 2's English Full-Attack, Gemini-1.5-Pro at 0.7906 is higher than DeepSeek-R1 at 0.6913, and this overall ranking does not distinguish by language. The authors state that language-aware alignment, contextual filtering, and automated safety review are needed.