Skip to content

Translation in progress Some entries still include text in the original language.

Original
arXiv· arXiv:2604.15717· Ki Sen Hung, Xi Yang, Chang Liu, Haoran Li, Kejiang Chen, Changxuan Fan, Tsun On Kwok, Weiming Zhang, Xiaomeng Li, Yangqiu Song·Indexed by this site · Originally publishedSelectedAttention41

Entering the Gray Zone: Domain Context Can Blur the Safety Boundaries of Large Language Models

Into the Gray Zone: Domain Contexts Can Blur LLM Safety Boundaries

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose the multi-turn jailbreak framework JARGON, which uses safety research papers as context and achieves ASR of 93%–100% against 7 frontier models on JailbreakBench.

Threat modelThe attacker interacts with the target LLM through multi-turn dialogue, first providing real security research materials (such as jailbreak papers) to establish an academic context, then rewriting harmful goals as case requests within the paper framework; the attack is automated by LLM agents, targets commercial and open-source conversational LLMs, and success is determined by a judge model scoring harmfulness.

Questions
LLM alignment requires models to be both helpful and harmless; the same knowledge can be used for good or ill, and the model must infer intent from context. The authors argue that domain context pushes queries into a "gray zone" where the refusal boundary is blurred, and that safety research context relaxes defenses across all harm categories.
Methods
The authors propose JARGON: using real safety research papers as context, it first conducts benign academic discussion to build rapport, then rewrites the harmful goal as a case request within the paper's framing, and generates multiple query variants, taking the highest harm score; successful trajectories are stored in a memory bank for retrieval in subsequent attacks.
Experiments and results
On JailbreakBench, JARGON achieves 93%–100% ASR on 7 models including GPT-5.2, the Claude-4.5 series, and Gemini-3-Flash, with an average of 99.0%, higher than the strongest baseline X-Teaming's 69.9%; on two medical models it is 100%. Activation analysis shows attack queries fall in an intermediate region between benign and harmful.
Limitations and future work
The authors note: other context formats (technical blogs, industry reports) were not tested; defenses only reduce ASR and do not fully eliminate the vulnerability; Knowledge Purification may overestimate harm scores. The experiments cover 10 general models and 2 medical models, with JailbreakBench and MedSafetyBench as benchmarks.
Experimental setupGPT-5.2、Claude-Sonnet-4.5 et al. · JailbreakBench、MedSafetyBench · ASR、Harm Score et al.
Evaluated models
GPT-5.2Claude-Sonnet-4.5Claude-Opus-4.5Gemini-3-FlashDeepSeek-V3.2Qwen3-235B-A22B-ITLLaMA-4-Scout-ITMeditron3-8BBioMistral-7BQwen3-8BLLaMA-3-8B-IT
Benchmarks
JailbreakBenchMedSafetyBench
Metrics
ASRHarm ScoreRelative Harm ScoreMMLUHellaSwagGSM8K
AI summaryand selection rationaleResearchers from institutions including the University of Science and Technology of China and the Hong Kong University of Science and Technology propose the jailbreak framework Jargon. The paper reports that its attack success rate on seven frontier models, including GPT-5.2, Claude-4.5, and Gemini-3, exceeds 93%, averaging 99%. The method uses security research papers as context together with multi-turn adversarial interaction, and its effect is markedly higher than that of existing methods such as PAIR, Crescendo, and X-Teaming. The paper observes that domain-specific professional context (such as chemistry) only selectively relaxes defenses for domain-related harms, whereas security research context triggers a general relaxation covering all harm categories. Activation-space analysis of Qwen3-8B shows that attack queries fall in a gray zone between benign and harmful inputs, where the model's refusal decisions become unreliable.

Researchers from institutions including the University of Science and Technology of China and the Hong Kong University of Science and Technology propose the jailbreak framework Jargon. The paper reports that its attack success rate on seven frontier models, including GPT-5.2, Claude-4.5, and Gemini-3, exceeds 93%, averaging 99%. The method uses security research papers as context together with multi-turn adversarial interaction, and its effect is markedly higher than that of existing methods such as PAIR, Crescendo, and X-Teaming. The paper observes that domain-specific professional context (such as chemistry) only selectively relaxes defenses for domain-related harms, whereas security research context triggers a general relaxation covering all harm categories. Activation-space analysis of Qwen3-8B shows that attack queries fall in a gray zone between benign and harmful inputs, where the model's refusal decisions become unreliable.

Why it was selectedThe authors demonstrate the use of policy guardrails and alignment fine-tuning to reduce the success rate of such context-based jailbreaks while preserving usefulness; safety training teams can draw on their data construction process.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors state that domain context blurs LLM safety boundaries, that safety research context can relax defenses across all harm categories, and they propose JARGON to exploit this.

    Core problem: the tension between helpful and harmless in LLM alignment makes models rely on context to infer intent, which the authors say creates an exploitable "gray zone".

    • Problem setting: alignment training teaches models when to refuse rather than forget knowledge; restricted knowledge remains encoded in the parameters. Legitimate professional users such as security researchers and pharmacologists must also be able to discuss sensitive topics, so the model cannot refuse across the board.
    • Shortcomings of existing methods: the authors say previous jailbreaks mostly used surface-level personas such as "I am a researcher", and as safety training has matured, modern LLMs can recognize such shallow disguises (Introduction).
    • Core observation: the authors propose a two-level vulnerability structure—Vertical Unlocking means domain context (e.g., chemistry) relaxes defenses only for domain-related harms; General Unlocking means safety research context (e.g., jailbreak papers) relaxes defenses across all harm categories (probing experiments in Figure 2).
    • Paper output: proposes the multi-turn jailbreak framework JARGON, performs activation-space and attention analyses to explain the mechanism, and finally explores two defenses: policy guardrails and alignment fine-tuning.
    • Threat model:
      • Goal: make the target LLM output specific content for the 10 categories of harmful behavior in JailbreakBench.
      • Knowledge and capability: the attacker possesses the full text of public safety research papers and interacts through multi-turn dialogue; attacks are fully automatically generated by an LLM agent, with no access to model internals.
      • Victim systems: commercial closed-source models (GPT-5.2, Claude-4.5 series, Gemini-3-Flash) and open-source models; the paper notes that the GPT-5 and Claude-4.5 series deploy front-end safety classifiers.
      • Success criterion: the judge model scores responses for harm on a 0–1 scale; a score in the [0.8, 1.0] interval is judged a success.

  2. What related research exists?

    Related work covers three lines: token-level, single-turn prompt, and multi-turn jailbreak; the authors state that these methods rely on shallow personas, whereas this paper leverages real domain context.

    Token-level and single-turn attacks

    • GCG (Zou et al., 2023) uses gradient optimization to find adversarial suffixes; AutoDAN (Liu et al., 2024a) uses genetic algorithms; AmpleGCG (Liao and Sun, 2024) and Jia et al. (2024) improve efficiency.
    • DAN (Shen et al., 2024) and its automated variants (Liu et al., 2024b; Jha and Reddy, 2023; Chao et al., 2023) embed bypass instructions directly in the prompt.

    Multi-turn jailbreaks

    • Crescendo (Russinovich et al., 2025) escalates requests gradually; ActorAttack (Ren et al., 2024) coordinates multiple agents; X-Teaming (Rahman et al., 2025) uses a multi-agent ensemble.
    • FITD (Weng et al., 2025) exploits psychological compliance, and RACE (Ying et al., 2025) exploits reasoning strategies; persona research shows that contextual framing affects compliance (Wu et al., 2025).
    • Paper critique: the authors state that these methods rely on shallow personas, which modern safety training is increasingly able to detect (final paragraph of Related Work).

    Defenses and evaluation

    • Defenses: fixed-taxonomy classifiers such as Llama-Guard (Inan et al., 2023), which the authors state struggle to generalize to out-of-distribution attacks; policy-driven guardrails (Li et al., 2025; OpenAI, 2025) support custom safety guidelines, and this paper implements its defenses based on them.
    • Benchmarks: JailbreakBench (Chao et al., 2024) is the main benchmark of this paper; MedSafetyBench (Han et al., 2024) is used for medical model evaluation.

    Baselines and positioning: the comparison baselines are PAIR and AmpleGCG (single-turn) and Crescendo, FITD, and X-Teaming (multi-turn), all evaluated on JailbreakBench. The authors position this paper's distinction as: rather than constructing shallow personas, it exploits the association of "technical expertise = legitimate intent" formed during training, using real safety research context to bypass defenses.

  3. How does the paper address this problem?

    JARGON has three stages: establish a security research context, build rapport through benign academic discussion, and rewrite harmful goals and generate variants to extract knowledge.

    JARGON operationalizes General Unlocking as an automated multi-turn attack: it first establishes a permissive context with real safety research material, then builds a cooperative pattern through benign discussion, and finally rewrites the harmful goal as an academic case request (Figure 3).

    Control Layer

    • The attack agent maintains the attack state and generates the next query each turn based on three types of information: the harmful goal g, the chat history, and a trajectory memory storing past successful trajectories.
    • It decides on its own when the target model is sufficiently "warmed up", switching from benign questions to attack queries embedded in the academic framing.

    Dialogue flow (two stages)

    • Rapport-Building: first presents safety research material P (a jailbreak paper or red-team blog), and for the first k turns (typically k=2) asks benign questions (requesting summaries, asking about methods), reinforcing the academic framing and establishing a cooperative dialogue pattern.
    • Attack: from turn k+1 onward, instead of asking g directly, it rewrites g as a case study within the paper's framing (e.g., requesting examples, hypothetical scenarios, comparative analysis), exploiting the established rapport to increase compliance.
    • Query variants: each attack generates V variants, half rewrites and half scenario reconstructions, evaluated in parallel with the response with the highest harm score selected.

    Judge and success criterion

    • The judge first extracts content relevant to the goal from the long dialogue history (Knowledge Purification), filters out the rapport portion, and then scores harm on a 0–1 scale.
    • A score above a preset threshold (the [0.8, 1.0] interval in the main experiments) is judged a successful attack.

    Trajectory Memory

    • Successful trajectories are stored in a buffer; when attacking a new goal, similar trajectories are retrieved by embedding cosine similarity as few-shot examples; the buffer starts from one seed trajectory and grows with successes.

    Implementation parameters

    • Five agents: Attacker and Optimizer use DeepSeek-Chat (t=0.3), Knowledge Purifier uses DeepSeek-Chat (t=0), Refusal Checker uses GPT-OSS-20B, and Judge uses DeepSeek-V3.2 (Appendix A.4.1).
    • Three nested loops: 3 retries × 2 trials each × 4 dialogue turns per trial; each attack generates 8 variants (4 rewrites + 4 scenarios); after two failed retries, few-shot examples are injected (Appendix A.4.2).
    • Appendix A.7 provides the full prompts for the Attacker, variant generation, judging, etc.

  4. What experiments did the paper conduct?

    On JailbreakBench, JARGON achieves ASR of 93%–100% against 7 frontier models, averaging 99.0%, higher than the strongest baseline X-Teaming's 69.9%.

    实验设置

    • 被测模型：主实验 7 个——GPT-5.2、Claude-Sonnet-4.5、Claude-Opus-4.5、Gemini-3-Flash、DeepSeek-V3.2、Qwen3-235B-A22B-IT、LLaMA-4-Scout-IT；另有医学模型 Meditron3-8B、BioMistral-7B（附录 A.3）和防御实验用的 Qwen3-8B、LLaMA-3-8B-IT。全部 temperature 0。
    • 基准：JailbreakBench，100 条有害提示词、10 个风险类别；医学评测用 MedSafetyBench 上 100 条采样查询。
    • 基线：单轮 PAIR、AmpleGCG；多轮 Crescendo、FITD、X-Teaming。
    • 指标与评审：主指标 ASR，由 DeepSeek-V3.2 按 0–1 打分，落入 [0.8, 1.0] 判成功；探测实验（Section 3）结果对三个种子取平均。
    • 默认配置：上下文为 X-Teaming 论文的 Abstract + Method 部分。

    主结果

    据 Table 2（ASR %，JailbreakBench）：

    Swipe horizontally to read the table

    方法GPT-5.2Claude-Sonnet-4.5Claude-Opus-4.5Gemini-3-FlashDeepSeek-V3.2Qwen3-235BLLaMA-4-Scout平均
    PAIR5012468153220.7
    AmpleGCG10512775222723.9
    Crescendo2223117973527347.6
    FITD5448249695734962.7
    X-Teaming59182294100999769.9
    JARGON9310010010010010010099.0
    • 作者称单轮攻击在防护强的模型上几乎完全失败（GPT-5.2 与 Claude-4.5 系列低于 10%），多轮基线也不稳定，如 X-Teaming 在 DeepSeek-V3.2 上 100% 但在 Claude-4.5 系列只有 18%–22%。
    • 作者称 GPT-5.2 与 Claude-4.5 系列对齐更强，归因于前端安全分类器与 GPT-5.2 的 safe completion 策略，但 JARGON 在其上仍达 93%–100%。

    探测实验：Vertical 与 General Unlocking

    • 测了什么：8 个危害主题 × 领域对齐论文摘要作上下文，对 Llama-4-Scout、Claude-Sonnet-3.7、Gemini-3-Pro 三个模型，另用单篇越狱论文统一作用于全部主题（Figure 2）。
    • 结果：热图呈对角线模式，领域对齐的语境-查询对分数更高（如病毒学论文配病毒合成查询得 0.78–0.96）；单篇越狱论文在所有主题上取得与领域对齐语境相当或更高的分数。
    • 作者解读：领域语境选择性放松领域相关防护（Vertical Unlocking）；安全研究语境因训练中与多类敏感主题的关联，触发跨域放松（General Unlocking）。

    机理分析：激活空间与注意力

    • 激活分析（Figure 5）：对 Qwen3-8B 第 24 层激活做 MDS 投影；作者称安全边界是渐变过渡区而非清晰分界线，rapport 阶段查询落在良性区，改写后的攻击查询移入良性与有害之间的"灰色地带"，且仍位于决策边界的良性一侧。
    • 注意力分析（Figure 6）：Qwen3-8B 上 20 条 JailbreakBench 查询（每类 2 条）三种条件对比：直接查询时敏感 token 注意力集中并触发拒答；加语境后直接查询仍保持高敏感 token 注意力；加语境并改写后敏感 token 注意力权重下降。作者称学术框架把注意力分散到语境元素上，降低了有害信号的显著性。

    防御探索

    • Policy 防护：用 gpt-oss-safeguard 加载自定义 policy（附录 A.8.4），输出分类与回复指导后传给目标模型。Qwen3-8B 的 ASR 从 100% 降到 61%；LLaMA-3-8B-IT 出现过度敏感，对所有查询一律拒答，论文未报告其 ASR（Table 3 标 –）。
    • 对齐微调：用约 900 条安全多轮对话（每篇语境论文约 300 条）以 LoRA 微调。Qwen3-8B 的 ASR 从 100% 降到 66%，LLaMA-3-8B-IT 从 83% 降到 74%；MMLU、HellaSwag、GSM8K 分数基本保持（Table 3）。
    • 作者解读：模型可以学会在灰色地带查询上保持 helpful 而不给出可操作的有害内容。

    其他消融与分析

    • 语境主题（Table 1）：越狱论文、防御方法、安全综述三类语境在 4 个模型上各测 20 条查询，平均 ASR 均超过 96%。
    • 语境长度（Figure 4）：Full Paper > Abstract+Method > Abstract 的 Relative Harm Score 在 3 个模型上均成立；主实验采用 Abstract+Method 平衡效果与 token 成本。
    • 查询变体（Table 4）：去掉变体后 GPT-5.2 从 93% 降到 54%，LLaMA-4-Scout-IT 从 100% 降到 79%，Claude-Sonnet-4.5 保持 100%。
    • 医学模型（Table 7）：Meditron3-8B 与 BioMistral-7B 在 MedSafetyBench 100 条查询上 ASR 均为 100%。
    • 评审偏差（Table 8，20 个样本）：LLM 评审相对人工高估 ASR（如对 GPT-5 高 25 个百分点），经 Knowledge Purification 后 GPT-5.2 评审对 GPT-5 从 90% 降到 70%，与人工的 65% 相差 5 个百分点以内。

  5. What are some points that can be further explored?

    The authors list three limitations: other context formats were not tested, defenses are incomplete, and Knowledge Purification may overestimate harm scores.

    Limitations and future directions noted by the authors

    • Context format coverage: General Unlocking was validated on attack papers, defense methods, and surveys, but other formats such as technical blogs and industry reports were not tested (Section 7).
    • Defenses are not complete: defenses reduced ASR but did not fully eliminate the vulnerability; the authors state that scaling up training data may yield stronger defenses (Section 7).
    • Risk of judge bias: Knowledge Purification may decontextualize statements, making them appear more harmful than in the original context and thereby inflating harm scores (Section 7).

    Experimental coverage

    • 12 models were tested in total: 7 frontier models in the main experiments, 2 medical models, Qwen3-8B and LLaMA-3-8B-IT for defense and mechanism experiments, and Claude-Sonnet-3.7 additionally in the probing experiments.
    • Benchmarks are JailbreakBench (100 prompts, 10 categories) and MedSafetyBench (100 sampled queries); the context-topic ablation tests only 20 queries per group on 4 models (Table 1).
    • The judge-bias study has a sample size of 20 (Table 8); probing experiments average over three seeds, and the main experiments do not report the number of repetitions.
    • Defenses were evaluated only on the two open-source models Qwen3-8B and LLaMA-3-8B-IT; defenses were not tested on commercial models.
    • The paper does not report specific numbers for the number of attack queries or token cost, mentioning only in the context-length analysis that Abstract+Method balances effectiveness and token cost.

  6. Summarize the main content of the paper

    The authors state that a security research context relaxes LLM defenses across categories, and JARGON exploits this to achieve 93%–100% ASR on 7 frontier models.

    A paper on multi-turn jailbreak attacks that also provides mechanism analysis and defense exploration.

    • Problem: The authors state that the tension between helpful/harmless in LLM alignment creates a "gray zone"—domain context selectively relaxes domain-related defenses (Vertical Unlocking), while security research context relaxes defenses across all harm categories (General Unlocking).
    • Method: JARGON uses real security research papers as context, first conducting benign academic discussion to build rapport, then rewriting harmful goals as case requests within the paper framework; each attack generates 8 variants and takes the highest harm score, with successful trajectories stored for retrieval; fully automated by LLM agents.
    • Main results: On JailbreakBench, ASR on 7 models including GPT-5.2, the Claude-4.5 series, and Gemini-3-Flash is 93%–100%, average 99.0%, versus the strongest baseline X-Teaming's average 69.9% (Table 2); medical models Meditron3-8B and BioMistral-7B both reach 100% on MedSafetyBench (Table 7).
    • Mechanism: Qwen3-8B activation projections show attack queries fall in an intermediate region between benign and harmful; attention analysis shows the academic framework reduces attention weights on sensitive tokens (authors' interpretation).
    • Defense: policy defenses reduce Qwen3-8B's ASR from 100% to 61%, and LoRA alignment fine-tuning reduces it to 66% while largely maintaining general capability, but LLaMA-3-8B-IT shows over-refusal under defenses (Table 3).
    • Authors' conclusion: Security research context constitutes a unique risk, and the authors hope the work drives context-aware alignment strategies; the authors acknowledge that defenses are incomplete and other context formats were not tested.
Read originalACL 2026
#Jailbreaks and attacks