X-Teaming: Adaptive Multi-Agent-Based Multi-Turn Jailbreak and Defense

1. 01What problem does this paper attempt to solve?

   The authors claim that multi-turn interactions can easily distribute malicious intent to bypass safety defenses; the paper proposes a collaborative multi-agent framework to achieve high success rates and diverse multi-turn red-teaming.

   Language models are vulnerable to distributed malicious intent attacks in multi-turn conversations, while existing red-teaming methods lack strategy diversity and dynamic adaptability.

   • Importance of Multi-turn Safety Risks: The authors state that single-turn safety defenses can already intercept single-prompt harmful inputs, but attackers can strategically distribute malicious intent across multi-turn interactions, and current systems struggle to effectively detect and prevent such progressive risks.
   • Limitations of Existing Red-Teaming Methods: Existing automated multi-turn attack methods often rely on a single attack strategy, fixed templates, or constrained interaction patterns, failing to simulate the strategic breadth of human red-teamers and lacking flexible adjustment when facing defenses.
   • Collaborative Multi-Agent Hypothesis: The authors believe that through collaboration among planning, attack execution, real-time verification, and prompt optimization, and by introducing text-gradient-based adaptive feedback, it is possible to systematically cover diverse attack scenarios and improve jailbreak effectiveness.
   • Threat Model:
     • Attack Goal: Induce the victim model to output fully violating content for harmful behaviors defined by HarmBench (Verifier score of 5).
     • Attacker Knowledge and Capabilities: The attacker operates in a black-box setting, sending queries to the target model via natural language conversation interfaces, dynamically obtaining model responses and engaging in multi-turn interactions (default maximum of 7 turns).
     • Victim Systems: Mainstream open-weight models (e.g., Llama-3 series, DeepSeek-V3) and frontier commercial closed-source models (e.g., GPT-4o, Claude series, Gemini-2.0-Flash).
   • Core Contributions: The paper proposes the adaptive multi-turn red-teaming framework X-Teaming and generates the multi-turn safety alignment dataset XGuard-Train, containing 30K interaction samples.

   Collapse this answer

2. 02What related research exists?

   Related work covers single-turn and multi-turn jailbreak attacks, agent prompt optimization, and interactive safety datasets; the authors note that existing methods have limited strategy diversity and multi-turn defense resources are constrained.

   Evolution of Single-turn and Multi-turn Jailbreak Attacks

   • Single-turn Jailbreak Methods: GCG (Zou et al., 2023) generates universal adversarial suffixes via gradient-based discrete optimization; PAIR (Chao et al., 2023) uses an attacker LLM to automatically generate and refine single-turn jailbreak prompts; CodeAttack (Jha & Reddy, 2023) generates adversarial code samples by manipulating token structures.
   • Multi-turn Attack Baselines: Crescendo (Russinovich et al., 2024) gradually guides conversations toward harmful topics, but the authors note its reliance on fixed seeds; ActorAttack (Ren et al., 2024) builds attack paths based on actor-network theory, but the authors note its constrained interaction patterns; FITD (Weng et al., 2025) uses the foot-in-the-door psychological technique, RACE (Ying et al., 2025) introduces reasoning tasks, and CoA (Yang et al., 2024b) uses semantic chain driving. The authors note these methods are mostly limited to narrow single strategies.

   Agent Architectures and Prompt Optimization

   • Defense and Attack Agents: Works like AutoDefense (Zeng et al., 2024b) explore using multi-agent collaboration for input filtering defenses, while this study uses agents for offensive red-teaming.
   • Adaptive Prompt Optimization: TAP (Mehrotra et al., 2023) uses tree search to generate attack prompts; TextGrad (Yuksekgonul et al., 2025) performs natural language prompt optimization via backpropagating text feedback. Unlike ActorAttack's self-talk, this work uses TextGrad for adaptive search based on the target model's actual responses.

   Interactive Safety Alignment and Resources

   • Single-turn and Multi-turn Evaluation Resources: Existing safety benchmarks and classifiers (e.g., HarmBench, WildGuard, SafetyBench) mainly target single-turn interactions, with fewer resources for multi-turn conversation safety evaluation.
   • Multi-turn Safety Datasets: SafeMTData (Ren et al., 2024) provides multi-turn safety fine-tuning data, but the authors note its small scale (1.4K samples) and generation from a framework with limited attack diversity, which can lead to overfitting to specific attack methods.

   Baseline Methods and Benchmarks

   • Comparison Baselines: Single-turn baselines include GCG, PAIR, CodeAttack; multi-turn baselines include RACE, CoA, Crescendo, FITD, ActorAttack.
   • Benchmarks and Datasets: Attack evaluation uses HarmBench (510 harmful behaviors); safety fine-tuning and generalization evaluation use WildJailbreak, SafeMTData, MultiJail, DAN, WildGuard, XSTest, Jailbreak Trigger, and general capability benchmarks MMLU, GSM8K, MATH, GPQA, HumanEval, HellaSwag, BIG-Bench-Hard.

   Positioning Differences The authors claim that, compared to existing multi-turn methods that rely on single strategies, fixed templates, or are not fully open-sourced, X-Teaming achieves higher strategy diversity and dynamic adaptability through multi-agent collaborative planning and text-gradient optimization, and contributes the open-source multi-turn safety alignment dataset XGuard-Train, which is 20 times larger in scale.

   Collapse this answer

3. 03How does the paper address this problem?

   The paper proposes the X-Teaming framework, where a planner, attacker, verifier, and optimizer collaborate in a two-stage process, enabling adaptive strategy planning and text-gradient prompt optimization.

   Overall, the authors propose an adaptive multi-turn red teaming framework X-Teaming, which divides the attack process into two stages—strategic attack planning and adaptive execution optimization—through the collaboration of four specialized agents: planner, attacker, verifier, and prompt optimizer.

   Problem Setting and Multi-Agent Architecture

   • System Composition: The framework includes four roles: planner P (using GPT-4o, sampling temperature 0.5), attacker A (using Qwen-2.5-32B-IT, sampling temperature 0.3), verifier V (using GPT-4o), and prompt optimizer (using Qwen-2.5-32B-IT).
   • Task Formalization: Given a target harmful behavior h, the system goal is to induce the target model to generate a response that fully complies with the harmful behavior within a maximum number of rounds (default maximum conversation rounds Tmax = 7).
   • History Representation: At round t, the system maintains multi-turn conversation history H_t = {(q₁, r₁, v₁), …, (q_t−1, r_t−1, v_t−1)}, where each entry contains the attack query, model response, and verification score.

   Stage One: Strategic Attack Planning

   • Diverse Plan Generation: The planner generates a candidate plan set S = {s₁, …, s_n} in iterative batches, representing the set of candidate attack strategies generated by the planner. The system sets the number of iterative batches Nsets = 5, with Nplans = 10 plans generated per batch, yielding a total of Ntotal = 50 candidate plans. Subsequent batches take the results of previous batches as context input, prompting the planner to generate non-repetitive strategies.
   • Four Elements of Plan Structure: Each plan includes a persona (Persona), specific context (Context), core strategy approach (Approach, such as educational angle, historical reference, story creation, etc.), and a round-by-round conversation plan (Conversation Plan).
   • Execution Subset Extraction: When actually initiating interaction, the system randomly draws a subset containing 10 plans from the 50 candidate plans for execution.

   Stage Two: Adaptive Attack Execution and TextGrad Optimization

   • Attack Advancement and Reasoning: The attacker generates queries based on the current plan step and conversation history, and uses thinking tags for strategy planning before outputting messages to simulate reasoning model behavior; the target model generates responses at sampling temperature 0.
   • Real-Time Verification Scoring: The verifier evaluates each round of the target model's responses on a scale of 1 to 5, where 1 indicates refusal or irrelevance and 5 indicates full compliance with the harmful behavior, while also providing a justification.
   • Text Gradient Prompt Optimization: If the current round's score does not decrease (i.e., the current score is greater than or equal to the previous score), the conversation proceeds to the next stage according to the original plan; if the score decreases, the system triggers the prompt optimizer, which uses the TextGrad method with the verifier's score and justification as loss function prompts to iteratively optimize the attack query via textual gradient descent, with at most 4 optimization retries per round.

   Dynamic Plan Revision and Termination Conditions

   • Adaptive Plan Extension: If a plan's preset steps are exhausted without reaching a score of 5 and the total number of conversation rounds has not reached the limit, the planner dynamically adds subsequent advancement stages based on the conversation history and feedback while maintaining the established persona and context.
   • Success Determination Mechanism: Once the target model's response in any round is rated 5 by the verifier, the jailbreak is deemed successful and the interaction process terminates; if the round budget is exhausted without achieving this, it is deemed a failure.

   Collapse this answer

4. 04What experiments did the paper conduct?

   X-Teaming achieves up to 98.1% ASR on the HarmBench test set and outperforms baselines in planning diversity, multilingual capability, and safety fine-tuning generalization.

   Experimental Setup

   • Target Models: Closed-source commercial models include GPT-4o, Claude-3.5-Sonnet, Claude-3.7-Sonnet, Gemini-2.0-Flash; open-weight models include Llama-3-8B-IT, Llama-3-70B-IT, Llama-3-8B-IT fine-tuned with SafeMTData, Deepseek V3, and Qwen-2.5-32B-IT.
   • Evaluation Benchmark and Scale: The main attack experiments use the HarmBench test set, containing 510 harmful behaviors across categories; a validation set is used for hyperparameter ablation studies.
   • Comparison Baselines: Single-turn baselines include GCG, PAIR, CodeAttack; multi-turn baselines include RACE, CoA, Crescendo, FITD, ActorAttack.
   • Metrics and Criteria: The primary metric is Attack Success Rate (ASR, %), i.e., the percentage of test cases that successfully induce the target model to produce violating behavior.
   • Evaluation Method: GPT-4o is used as the primary verifier for scoring (1 to 5, with 5 indicating a successful attack), and scoring consistency is checked against the official HarmBench classifier and LlamaGuard 3.
   • Experimental Parameters and Budget: Target model sampling temperature is set to 0; default configuration has a maximum of 7 conversation turns, 10 attack plans, and up to 4 TextGrad optimization attempts.

   Main Results

   The table below shows the attack success rates of each method on the HarmBench test set (from Table 2):

   Swipe horizontally to read the table

   Target Model	GCG	Crescendo	FITD	ActorAttack	X-Teaming (Ours)
   GPT-4o	12.5	46.0	84	84.5	94.3
   Claude-3.5-Sonnet*	3.0	50.0	Not reported	66.5	67.9*
   Gemini-2.0-Flash	Not reported	Not reported	Not reported	42.1	87.4
   Llama-3-8B-IT	34.5	60.0	93	79.0	85.5
   Llama-3-70B-IT	17.0	62.0	Not reported	85.5	84.9
   Llama-3-8B-IT (SafeMTData)	Not reported	12.0	Not reported	21.4	91.8
   Deepseek V3	Not reported	Not reported	Not reported	68.6	98.1

   *Note: According to the Table 2 note, under the full configuration (50 plans, 5 TextGrad attempts, 10 turns), Claude 3.5 Sonnet reaches 67.9%, and Claude 3.7 Sonnet reaches 96.2%. The table omits single-turn methods PAIR, CodeAttack, and multi-turn methods RACE, CoA.

   • Surpassing existing single-turn and multi-turn baselines: The authors claim X-Teaming achieves higher ASR on most tested models, with DeepSeek-V3 at 98.1% and GPT-4o at 94.3%.
   • Breaking through multi-turn safety fine-tuned models: On Llama-3-8B-IT specifically fine-tuned with SafeMTData for multi-turn safety, X-Teaming achieves 91.8% ASR, while ActorAttack only reaches 21.4%.
   • Consistent improvement with minimal parameter gaps: X-Teaming outperforms comparison methods on most models, only slightly lower than ActorAttack on Llama-3-70B-IT (84.9% vs. 85.5%).

   Attack Efficiency and Resource Overhead

   • What was measured: Statistics on the average number of turns, plans, optimization attempts, and generated tokens required for successful attacks (from Table 7, Table 8).
   • Results: X-Teaming requires on average only about 3.5 plans, 0.6 TextGrad optimizations, and 4.3 conversation turns for a successful jailbreak; in a constrained alignment comparison experiment (10 plans/roles, same token budget, both using Qwen-2.5-32B as the attack model against GPT-4o), X-Teaming achieves 94.6% ASR, while ActorAttack achieves 75.7%.
   • Authors' interpretation: The authors claim X-Teaming replaces closed-source commercial models with the open-source Qwen-2.5-32B as the attacker, achieving higher attack success rates while ensuring lower invocation costs.

   Attack Strategy Diversity Evaluation

   • What was measured: Using MiniLMv2 to compute embedding cosine similarity diversity scores among attack plans (roles, contexts, methods) and among attack queries (from Figure 2, Figure 4).
   • Results: X-Teaming achieves an average diversity score of 0.702 at the plan level, higher than ActorAttack's 0.278; at the specific attack query level, it achieves an average score of 0.466, higher than ActorAttack's 0.288.
   • Authors' interpretation: The authors claim X-Teaming demonstrates higher strategy diversity, enabling exploration of a broader range of attack scenarios.

   Safety Defense Fine-tuning Based on XGuard-Train

   • What was measured: Safety alignment fine-tuning of Llama-3.1-8B and Qwen-2.5-7B based on XGuard-Train (30K samples), evaluated on multi-turn attack, single-turn safety, and general capability benchmarks (from Table 5).
   • Results: Against X-Teaming attacks, the XGuard-Train fine-tuned Llama-3.1-8B model reduces ASR to 52.2% (SafeMTData: 93.7%); Qwen-2.5-7B achieves 8.7% ASR against Crescendo attacks (SafeMTData: 22.6%). Performance remains stable on capability benchmarks such as MMLU, GSM8K, and MATH.
   • Authors' interpretation: The authors claim that interactive safety data generated by X-Teaming provides defense generalization across attack frameworks without compromising general language understanding and reasoning capabilities.

   Additional Ablations and Analyses

   • Hyperparameter ablation (Table 10): On the HarmBench validation set, increasing TextGrad iterations from 3 to 4, turns from 6 to 7, and plans from 5 to 10 improves ASR on the SafeMT fine-tuned model from 78.0% to 95.1%.
   • Plan count analysis (Figure 3a): With 7 turns fixed and TextGrad disabled, increasing plans from 10 to 40 raises ASR from 70.7% to 97.6%, and remains at 97.6% when increased to 50.
   • Conversation turn analysis (Figure 3b): With 10 plans fixed and TextGrad disabled, increasing turns from 2 to 8 raises ASR from 19.5% to 92.7%, but drops to 87.8% when increased to 10.
   • TextGrad optimization analysis (Figure 3c): Increasing optimization attempts from 0 to 2 raises ASR from 70.7% to 97.6%, and stabilizes at 90.2% for 3 and 4 attempts.
   • Additional model testing (Table 11): On the HarmBench test set, Qwen-2.5-32B-IT achieves 99.4% ASR against X-Teaming.
   • Multilingual generalization evaluation (Table 9): On the MultiJail benchmark with GPT-4o as the target model, X-Teaming achieves 64%, 56%, and 52% ASR in Chinese, Arabic, and Bengali, respectively.

   Negative Results and Exception Analysis

   • Semantic category resistance (Table 6): The cybercrime category achieves 100% ASR on almost all models, while among the 3,629 failed attacks analyzed, chemical weapons, extreme violence, and extreme hate content maintain 0% success rates across all tested models.
   • Performance decline with increased turns (Figure 3b): ASR declines when conversation turns exceed 8; the authors' analysis indicates that the attack model tends to deviate from the preset persona and context in long conversations, triggering refusal responses from the target model.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors point out limitations such as long-context dilution degrading attack quality and potential verifier bias; experiments are mainly conducted on specified benchmarks and fixed turn configurations.

   Limitations and Future Directions Noted by the Authors

   • Long-context Dilution: The authors note that after more than 8 turns, the attacker model tends to deviate from the initially set persona and context, degrading query quality and triggering target model refusals (Section 3.3).
   • Potential Verifier Evaluation Bias: The authors note that relying on GPT-4o as the primary evaluator may introduce LLM evaluation bias, despite its 84.50% agreement with the official classifier (Sections 3.1, 3.2).
   • Overfitting Risk of Existing Defense Data: The authors note that small-scale multi-turn safety fine-tuning data can easily overfit to specific attack methods, requiring larger and more strategy-diverse datasets (Section 5).
   • Diminishing Returns of Prompt Optimization: The authors note that TextGrad iterations beyond 2 show diminishing returns and plateau; the 3rd and 4th attempts are unnecessary in most scenarios (Section 3.3).
   • Open-Source Plans for Model Weights and Training Schemes: The authors plan to release weight checkpoints of safety-trained models and reproducible training schemes to support community research on multi-turn safety alignment (Section 6).
   • Responsible Access and Dual-Use Risk Control: The authors clarify the dual-use nature of multi-turn vulnerability attack techniques in the ethics statement, emphasizing the need to restrict usage to defense and research (Ethics Statement).

   Experimental Coverage

   • Target Model Scope: Main experiments cover 7 target models (GPT-4o, Claude-3.5-Sonnet, Gemini-2.0-Flash, Llama-3-8B-IT, Llama-3-70B-IT, SafeMTData fine-tuned Llama-3-8B-IT, Deepseek V3), with supplementary tests on Claude-3.7-Sonnet and Qwen-2.5-32B-IT.
   • Jailbreak Evaluation Benchmark Scope: Main attack experiments are conducted on the HarmBench test set of 510 harmful behaviors; multilingual evaluation covers Chinese, Arabic, and Bengali on MultiJail.
   • Conversation Turns and Strategy Budget: Core experiments set a maximum of 7 conversation turns, up to 10 attack plans, and up to 4 TextGrad optimizations per turn; the extended configuration allows up to 10 turns.
   • Safety Fine-tuning Training Data Scale: From WildJailbreak, 10,000 harmful behaviors are sampled to construct the XGuard-Train dataset with 30,695 multi-turn interactions, used for LoRA fine-tuning of 2 models (Llama-3.1-8B and Qwen-2.5-7B).
   • Unreported Information: The paper does not report GPU memory overhead or specific latency data for multi-turn interactions with each target model during attacks.

   Collapse this answer

6. 06Summarize the main content of the paper

   The paper proposes an adaptive multi-agent framework X-Teaming to address weak multi-turn jailbreak defenses, achieving high attack success rates on mainstream models and releasing a large-scale safety dataset.

   • Positioning and Core Problem: This study addresses the vulnerability of large language models to distributed malicious intent attacks in multi-turn conversations and the lack of strategy diversity and adaptability in existing automated red-teaming methods, proposing a collaborative multi-agent automated multi-turn jailbreak framework, X-Teaming.
   • Core Method Design: The framework uses a planner to generate candidate plans containing personas, contexts, and multi-step advancement strategies; an attacker executes interactions; a verifier provides real-time 1-to-5 scoring; when scores decrease, an optimizer invokes TextGrad for text-gradient prompt optimization; if unsuccessful, it adaptively expands subsequent advancement plans.
   • Attack Success Rate Performance: On the HarmBench test set, X-Teaming achieves 98.1% ASR on DeepSeek-V3, 94.3% on GPT-4o, and 91.8% on SafeMTData fine-tuned Llama-3-8B-IT; under the extended configuration, it reaches 96.2% ASR on Claude 3.7 Sonnet (Table 2).
   • Strategy Diversity and Efficiency: Compared to the baseline ActorAttack, X-Teaming improves the average plan-level diversity score from 0.278 to 0.702, and successful attacks require on average only about 3.5 plans, 0.6 optimizations, and 4.3 interaction turns; under equal resource budget tests, it achieves 94.6% ASR on GPT-4o, higher than ActorAttack's 75.7% (Table 7, Table 8, Figure 2).
   • Interactive Safety Alignment Contribution: Based on this framework, the open-source safety fine-tuning dataset XGuard-Train, containing 30K multi-turn dialogues, is constructed; fine-tuned Llama-3.1-8B reduces ASR against X-Teaming attacks from 93.7% to 52.2%, while maintaining general capabilities and demonstrating defense generalization across attack frameworks (Table 5).
   • Authors' Conclusions and Implications: The authors believe that cyber exploitation and social manipulation are most vulnerable to multi-turn attacks, while content like chemical weapons and extreme violence is most robust; multi-turn red-teaming and dedicated safety datasets are crucial to fill the defense blind spots of current single-turn-only safety measures.

   Collapse this answer

Related papers