Multi-turn jailbreak attacks on multimodal large language models and FragGuard defense

Can multi-turn multimodal prompts bypass the basic safety guardrails of MLLMs, and how to intercept them without training.

• Scenario: The authors state that MLLMs have been used for VQA, image captioning, visual document extraction, and multimodal reasoning, and that jailbreaks, system prompt extraction, and data poisoning are also increasing. Multi-turn jailbreaks of text LLMs have improved success rates, but multi-turn prompt jailbreaks of MLLMs still lack systematic analysis.
• Existing approaches: The MLLM jailbreaks the authors list include related visual content, inconsistent prompts, and typographic images; defenses include modifying architecture, system prompts, and removing visual content. The authors believe these defenses may be insufficient under multi-turn jailbreaks and lack systematic evaluation of open-source and closed-source production models.
• Observation: The authors state that direct harmful requests are rejected by guardrails; if the conversation starts with a lower-harm request and then gradually adds hypothetical scenarios and harmful requests in images, the model is more likely to give a jailbroken response.
• Our approach: The authors propose the framework MJAD-MLLMs, which includes multi-turn jailbreak, a segment-level multi-LLM defense FragGuard that requires no training or fine-tuning, and multi-LLM review with human verification.
• Threat model:
  • Goal: Generate a jailbroken response R* at the target turn i, i.e., bypass safety alignment and elicit forbidden or harmful content.
  • Knowledge: The model Fθ is treated as a black box; the attacker cannot access internal parameters, modify weights, or directly manipulate behavior.
  • Capabilities: Can interact over multiple turns; each turn's input is a prompt pair consisting of an adversarial text query and an adversarial visual prompt, with history containing previous prompt pairs and responses.
  • Victim system: An LVLM, a subclass of MLLM, containing a vision module, a connector, and a text module with basic safety guardrails.

Multimodal jailbreak

• MM-SafetyBench (Liu et al.) uses images with typographic text for MLLM safety evaluation; this paper uses it as a dataset and as an attack baseline in Table 1.
• FigStep (Gong et al., 2025) writes harmful requests as typographic steps; SI-Attack (Zhao et al., 2025) shuffles images and text queries. Both are baselines in Table 1.
• The paper also describes system prompt jailbreaks (Ideator), ASCII art (ArtPrompt, Jiang et al., 2024), and hiding malicious content in images (Li et al., 2024), without providing reproduction numbers for each of these works.

Multi-turn jailbreak of text LLMs

• Crescendo (Russinovich et al., 2024), Foot-in-the-door (Weng et al.), Chain of Attack (Yang et al., 2024), Many-shot jailbreaking (Anil et al., 2024) study LLM jailbreaks under multi-turn prompts.
• Red Queen (Jiang et al., 2025) and multi-turn context jailbreak (Sun et al., 2024) are used in Insights to discuss weakened intent recognition after multiple turns; the paper does not rerun them as MLLM baselines.
• SafeMT (Zhu et al.) studies MLLM response safety under multi-turn attacks. The authors state that it fails to achieve high attack success rates under open-source LLM (Gemma-3) review.

MLLM defenses

• AdaShield (Wang et al., 2024) uses adaptive shield prompts; Eyes Closed, Safety On (Gou et al., 2024) replaces the vision module with text at prompt time; MLLM-Protector (Pi et al., 2024) does prompt detoxification.
• There are also agent defenses (AutoDefense, Zeng et al., 2024; X-Guard, Upadhayay et al., 2025), JBShield (Zhang et al., 2025), and hierarchical editing (Zhao et al., 2024). The authors state that these techniques may not be sufficient against emerging jailbreaks including multi-turn attacks.

Baselines and benchmarks

• Attack baselines are MM-SafetyBench, SI-Attack, and FigStep; the benchmark is the 13 categories of prohibited scenarios from MM-SafetyBench. The defense comparison is the authors' own full-response multi-LLM defense, i.e., without response splitting.

The authors position this paper as filling the gap of systematic analysis of multi-turn jailbreaks for MLLMs, and use training-free FragGuard to mitigate text-based jailbreaks including multi-turn ones.

The framework MJAD-MLLMs uses a three-turn conversation with typography-based images to induce jailbreak, then uses FragGuard to split the response, with multiple text LLMs taking the maximum toxicity score to decide whether to keep the original response or replace it with a safe refusal.

Problem Setting

• Models: The paper uses LVLM to represent MLLM. The vision module converts images into embeddings, the connector maps them into the latent space of the text module, and the text module combines visual features with text to generate responses.
• Formalization: The visual prompt is V, the text is a token sequence Q, and the output R follows the conditional distribution PFθ(R|Q,V). Jailbreaking aims to construct an adversarial pair (Qadv, Vadv) that makes the model generate a jailbroken sequence R*.
• Multi-turn condition: Generation at turn i is also conditioned on history H(i-1). Jailbreaking refers to bypassing safety alignment with adversarial prompts to elicit prohibited or harmful responses; the harmfulness reference is OpenAI prohibited scenarios.

Multi-turn Attack

• Visual carrier: The image content corresponds to prohibited scenarios; harmful requests or keywords are placed at the bottom of the image using typography, resembling a caption. Images come from MM-SafetyBench, generated by Stable Diffusion from extracted key phrases, with the phrase overlaid.
• Three-turn structure: Turn 1 is a benign question to enter the conversation; the authors do not evaluate harmfulness at this turn. Turn 2 asks the model to imagine the content and actors in the image within a hypothetical scenario. Turn 3 asks the model to respond to the adversarial request written in the image via typography.
• Rationale given by the authors: Direct harmful requests would be rejected by guardrails; gradually transitioning from lower-harm requests can keep the model cooperative. Appendix C has success and failure examples; the authors state that the original multi-turn adversarial prompts are intentionally omitted from the figures.

FragGuard

• Splitting: The response R* is split into several segments by a fixed token length. Appendix B states that each segment is 400 tokens in all defense experiments.
• Scoring models: Each segment is sent to three types of text LLMs: reasoning model o1, general model Gemini-2.5-Flash-lite, and open-source model LLaMa-3 (70B). Model Dj gives a toxicity score Tj,k for segment k.
• Aggregation: The final score is the maximum over all defense models and all segments, T_final=max_jmax_kT_j,k. This formula represents taking the most conservative toxicity score. If it exceeds a preset threshold τ, the original response is replaced with a predefined safe refusal; otherwise R* is returned. The paper does not give τ.
• Deployment: The defense models use max tokens of 50 and temperature of 0.3. Appendix B says the authors wrote system prompts for these models to evaluate harmfulness on a 1–5 scale, but the full prompt text is not given. The baseline full response defense does not split and scores the entire response as one unit.

Success Determination

• ASR and ATS: GPT-4 and Gemini-2.5-flash score toxicity on a 1–5 scale according to OpenAI prohibited scenarios, taking the higher score; a score greater than or equal to 4 counts as success, with the threshold following the cited SI-Attack paper. ASR is the proportion of successful samples, and ATS is the average toxicity score.
• RR: The proportion of adversarial queries that the model explicitly refuses when the defense is enabled. The authors state that higher RR means stronger defense, and note that ASR and ATS themselves do not directly indicate whether the defense refuses.
• Second judge set and human evaluation: Mistral-7B and Gemini-2.5-Flash are also used for toxicity scoring; 5 samples are randomly drawn from each category for human evaluation.

Experimental Setup

• Target models: Open-source are LLaVa-1.6 (7B), LLaVa-1.6 (13B), and Qwen-2-7B as stated in the main text; closed-source are GPT-4o (API date 2024-11-20) and Gemini-2.0-Flash. In tables they are abbreviated as LLaVa-7B, LLaVa-13B, and Qwen-7B. The model cards in the references are llava-v1.6-mistral-7b-hf, llava-1.5-13b-hf, and Qwen2-VL-7B-Instruct.
• Data: MM-SafetyBench, 13 OpenAI prohibited scenarios, 1680 items in total with uneven counts per category; the authors take 40 items per category. Categories include Illegal-Activity, HateSpeech, Malware-Generation, Physical-Harm, Economic Harm, Fraud, Sex, Political Lobbying, Privacy-Violence, Legal-Opinion, Financial Advice, Health-Consultation, Gov-Decision.
• Attack and defense configuration: Fixed 3 turns, turn 1 is not evaluated for harmfulness. Segment length is 400 tokens. Primary defense LLMs are o1, Gemini-2.5-Flash-lite, and LLaMa-3 (70B); one experiment replaces o1 with Mistral-7B. Hardware is 4 NVIDIA A100s with 40GB each. Target model max tokens are all 768; temperature, top p, and repetition penalty are respectively: LLaVa-7B 0.3/1.0/1.0, LLaVa-13B 0.7/0.9/1.05, Qwen-7B 0.3/0.9/1.1, GPT-4o 1.0/1.0/0.0, Gemini-2.0-Flash 1.0/0.95/0.0 (Table 7).
• Baselines: MM-SafetyBench, SI-Attack, and FigStep, compared only on LLaVa-7B and GPT-4o. The authors state that the first two baselines for LLaVa-7B and SI-Attack for GPT-4o are quoted from the original papers; FigStep and MM-SafetyBench on GPT-4o are implemented by the authors on the same dataset.
• Judges: Primary judge is GPT-4 and Gemini-2.5-flash, taking the max, with greater than or equal to 4 counting as success, and ATS is reported; RR is additionally reported for defense. The second judge set is Mistral-7B and Gemini-2.5-Flash. Human evaluation is 5 per category, covering only LLaVA-7B and GPT-4o. The paper does not report the number of repetitions or τ.
• Benign probe: Figure 6 uses the ChatGPT web interface for VQA on a benign image.

Main Results

The authors state that under the primary judge, the highest open-source ASR is 91.5% (LLaVa-7B) and the highest closed-source is 82.3% (Gemini-2.0-Flash); the latter number is not labeled with a specific turn. Table 1 Turn 3 ASR is 91.50% for LLaVa-7B and 77.31% for GPT-4o. The remaining point-by-point readings in Figure 2 are not listed in the main text. The table below shows the full-model attack results under the second judge set (from Table 2; the header reads Gemini 2.5-Falsh).

Swipe horizontally to read the table

Model	Turn2 ASR	Turn3 ASR	Turn2 ATS	Turn3 ATS
LLaVa-7B	73.08%	91.35%	3.75	4.62
LLaVa-13B	24.42%	16.54%	2.30	2.17
Qwen-7B	25%	52.31%	1.98	3.34
GPT-4o	35.19%	67.50%	2.79	3.56
Gemini-2.0-Flash	43.71%	68.09%	3.04	3.58

• The authors state that under the primary judge, turn 3 ASR is higher than turn 2 for all studied models; LLaVa-13B is lower than LLaVa-7B at both turns, which the authors suggest indicates that a larger model in the same family has stronger safety alignment under the same attack. Among closed-source models, the authors state that Gemini-2.0-Flash is more easily compromised than GPT-4o at both turns.
• Table 2 does not always show higher turn 3: LLaVa-13B ASR goes from 24.42% to 16.54%, and ATS from 2.30 to 2.17. In the same table, GPT-4o Turn 3 ASR is 67.50%, lower than the 77.31% under the primary judge in Table 1.
• Table 1 baseline ASR: MM-SafetyBench is 72.14% for LLaVa-7B and 12.34% for GPT-4o; SI-Attack is 62.68% and 68.57%; FigStep is 62.30% and 19.81%. The authors state that the paper's Turn 3 is higher than these baselines.

Defense Comparison

• Primary judge, Turn 3 (Table 3): LLaVa-7B attack ASR/ATS is 91.50%, 4.64; full response is 4.42%, 1.19; FragGuard is 0.96%, 1.06. GPT-4o attack is 77.31%, 3.82; full response is 13.46%, 1.14; FragGuard is 8.08%, 1.12. In Section 4.4.2 the authors state ASR is reduced by about 90% and 68%, and the conclusion states about 89% for open-source and about 68% for closed-source; the authors also state that relative to full response, ASR and ATS improve by about 4 and 5. The original numbers above can be verified in the table.
• Replacing defense model and judge (Table 4, Table 5): o1 is replaced with Mistral-7B, and the judge is changed to Mistral-7B and Gemini-2.5-Flash. FragGuard Turn 3 ASR is 1.15% for LLaVa-7B, 4.42% for LLaVa-13B, 7.31% for Qwen-7B, 0.00% for GPT-4o, and 0.58% for Gemini-2.0-Flash. Under the same setting, full response Turn 3 ASR is 6.35%, 6.35%, 15.00%, 3.50%, and 35.01%. Table 5's header still reads Full Response Defense; here it is attributed to FragGuard according to the section title.
• RR and exceptions: The authors state that in Figure 5, FragGuard's RR is higher than full response at both turn 2 and turn 3, but the figure's numbers are not given. Table 5 Turn 3 RR is 90.00% for LLaVa-7B, 13.46% for LLaVa-13B, 46.54% for Qwen-7B, 80.38% for GPT-4o, and 88.78% for Gemini-2.0-Flash. In Table 4, Qwen-7B and Gemini-2.0-Flash full response ASR goes from 11.35% and 29.21% at turn 2 to 15.00% and 35.01% at turn 3. In Table 5, LLaVa-13B FragGuard RR goes from 24.23% to 13.46%. The authors state that in Figure 4, ATS decreases across all 13 categories for both model types, but per-category numbers are not given in the main text. Appendix C has success and failure examples for attack and defense, without accompanying scores.

Human and LLM Evaluation

• Table 6 has 5 items per category. Defense is the same for both: LLaVA-7B ASR 1.53%, ATS 1.07; GPT-4o ASR 6.15%, ATS 1.75.
• Attack: LLaVA-7B LLM/human ASR is 87.69%/89.23%, ATS is 4.47/4.53; GPT-4o is 73.84%/69.23%, ATS is 3.73/3.47. The authors state ASR deviation is about 2.5% and 4.5%, and ATS deviation is about 0.06 and 0.4; when compared with the table, the original numbers in the table take precedence. The authors believe multi-LLM evaluation can improve evaluation quality, but more analytical methods are still needed.

Other Ablations and Analysis

• Segment length is fixed at 400 tokens; the paper does not report scans of other lengths or τ.
• Figure 6: The authors state that FragGuard does not rewrite that benign text, and infer from this that it does not hinder utility such as VQA; this is a single instance.
• RQ1: The authors believe that after multiple turns, the model finds it harder to recognize overall malicious intent, and latent representations shift toward helpfulness; this is an explanation, not additional experimental numbers.
• RQ4: The authors state that FragGuard can transfer to Audio-Flamingo, AudioPaLM, and to FigStep and SI-Attack; the paper does not report numbers for these transfers.
• Figure 3 is a turn 3 sample for LLaVa-7B. The authors state it contains executable operation details and manually highlighted them in red; the toxicity score in the figure is not transcribed in the main text.

Limitations and future directions noted by the authors

• The paper has no separate Limitations section and does not explicitly frame any result as its own limitation.
• Section 7: The authors plan to develop query optimization to automatically generate adversarial prompts.
• Section 7: The authors plan to analyze the susceptibility of commercial and reasoning MLLMs under the proposed jailbreak.
• Section 7: The authors plan to develop defenses against automated jailbreaks.

Experimental coverage

• Attack targets are LLaVa-7B, LLaVa-13B, Qwen-7B, GPT-4o, Gemini-2.0-Flash, 5 in total; the baseline comparison in Table 1 only includes LLaVa-7B and GPT-4o.
• Data is from MM-SafetyBench with 13 categories, 40 items per category, 1680 items in total; human evaluation uses 5 items per category, and only for LLaVA-7B and GPT-4o (Table 6).
• Defense results in Table 3 only include Turn 3 for LLaVa-7B and GPT-4o; defense numbers for all five models are in Table 4 and Table 5, and both the defense LLM and the review LLM are changed.
• Attacks are fixed at 3 turns, and turn 1 is not evaluated for harmfulness. The paper does not report repetition count, threshold τ, or numerical comparisons with cited defenses such as AdaShield.
• The benign task is a one-time ChatGPT web VQA in Figure 6. The authors discuss possible extensions to audio-to-text models and single-turn attacks but do not report numbers for these settings.

MJAD-MLLMs simultaneously provides black-box multi-turn jailbreak and training-free defense for LVLMs. The authors are concerned that although the text module has basic guardrails, multi-turn multimodal prompts may still elicit prohibited content.

• Attack: The image content corresponds to OpenAI prohibited scenarios, with harmful keywords placed at the bottom using typography. The conversation is fixed at 3 turns: first a benign visual question, then an entry into a hypothetical scenario, and finally a request pointing to the adversarial request in the image. Turn 1 is not evaluated for harmfulness.
• Defense: FragGuard splits the response into 400-token segments, scores toxicity with o1, Gemini-2.5-Flash-lite, and LLaMa-3 (70B), and takes the maximum; if it exceeds an unpublished threshold τ, it replaces the response with a safe refusal. The baseline is a non-split full response defense.
• Attack numbers: Under the primary judge, Table 1 Turn 3 ASR is 91.50% for LLaVa-7B and 77.31% for GPT-4o; the authors state the highest closed-source is 82.3% (Gemini-2.0-Flash). In the same table, the highest baseline is SI-Attack on GPT-4o at 68.57%, and MM-SafetyBench on LLaVa-7B is 72.14%.
• Judge differences: With Mistral-7B and Gemini-2.5-Flash instead, LLaVa-7B Turn 3 ASR is 91.35% and GPT-4o is 67.50%; LLaVa-13B goes from 24.42% at turn 2 to 16.54% at turn 3 (Table 2).
• Defense numbers: In Table 3, FragGuard reduces the above two Turn 3 ASRs to 0.96% and 8.08%. The authors state reductions of about 90% and 68%, and the conclusion separately states about 89% for open-source and about 68% for closed-source. With a different defense model and judge, GPT-4o Turn 3 ASR is 0.00% (Table 5).

The authors believe multi-turn weakens the model's recognition of overall malicious intent, and believe that in one benign VQA, FragGuard does not rewrite the response. The authors plan to automatically generate adversarial prompts, then analyze commercial and reasoning-type MLLMs, as well as defense against automated attacks.