Skip to content

Jailbreak Connectivity: Toward Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreaks

Jailbreak Connectivity: Towards Diverse, Transferable, and Universal Multimodal Large Language Model Jailbreak

arXiv: Adversarial Attacks, Backdoors & Poisoning· arXiv:2610.38091· Xu Zhang, Ziqing Hu, Shuo Han, Ren WangIllinois Institute of Technology; Perplexity AI; University of Illinois ChicagoIndexed by this site · 21 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose the multimodal jailbreak framework JC, with white-box average ASR of 84.01%–97.95% on MM-SafetyBench and black-box transfer to GPT-5.6 with average ASR of 41.94%.

Threat modelThe attacker manipulates the image input (adding a perturbation ∥xp−x∥∞≤ϵ) to make the target MLLM answer harmful questions; single-turn interaction; two settings, white-box (access to model parameters and gradients) and black-box (query only, no internal knowledge), where in the black-box setting images are generated with an open-source base model and transferred to closed-source models.

Questions
Existing multimodal jailbreak methods only optimize isolated adversarial examples, which the authors argue leads to insufficient diversity, poor cross-model transferability, and a lack of universality across multiple harmful queries, making it difficult to expose the true risk surface of MLLMs.
Methods
Proposes Jailbreak Connectivity (JC): hypothesizes that jailbreak effectiveness forms a connected low-loss region in image space, connects two jailbreak endpoints with a quadratic Bézier curve and samples along the path; then uses a CLIP-backbone safety classifier and jailbreak success predictor to guide transfer, and extends the objective to a harmful output distribution to achieve universal jailbreak.
Experiments and results
On the 13 scenarios of MM-SafetyBench, JC's white-box average ASR on the three open-source models is 84.01%–97.95%; with Qwen3-VL-8B-Instruct as the base model, black-box transfer to GPT-5.6 and Gemini 3.1 Pro yields average ASR of 41.94% and 47.05%, higher than FigStep's 17.25% and 19.71%.
Limitations and future work
The authors note that PPL only reflects fluency under a fixed evaluation model and does not directly measure harmfulness (Conclusion). The experiments cover 5 target models, MM-SafetyBench and AdvBench, with harmful corpus taken from AdvBench's 100 sentences; details such as proxy classifier fidelity and hyperparameter sensitivity are stated by the paper to be in the supplementary material, with no numbers given in the main text.
Experimental setupQwen3-VL-8B-Instruct、InternVL3.5-8B et al. · MM-SafetyBench、AdvBench · ASR、PPL
Evaluated models
Qwen3-VL-8B-InstructInternVL3.5-8BGemma-3-12B-ITGPT-5.6Gemini 3.1 Pro
Benchmarks
MM-SafetyBenchAdvBench
Metrics
ASRPPL
AI summaryThe researchers propose a visual jailbreak attack framework, Jailbreak Connectivity (JC), targeting multimodal large language models, achieving average attack success rates of 84.01%, 85.77%, and 97.95% on MM-SafetyBench against Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively, exceeding the strongest baseline by up to 68.36 percentage points. JC optimizes continuous adversarial paths based on the observation that effective adversarial inputs can be connected through low-loss regions in image space: path sampling generates diverse jailbreak images, a lightweight proxy classifier guides cross-model transfer, and a distribution-level objective enables universal attacks against different harmful queries. In black-box evaluations, JC's attack success rates on GPT-5.6 and Gemini 3.1 Pro are 24.69 and 27.34 percentage points higher than FigStep, respectively. The authors state that these findings reveal structural vulnerabilities in the safety alignment of multimodal large language models and can be used for more comprehensive safety evaluation.

The researchers propose a visual jailbreak attack framework, Jailbreak Connectivity (JC), targeting multimodal large language models, achieving average attack success rates of 84.01%, 85.77%, and 97.95% on MM-SafetyBench against Qwen3-VL-8B-Instruct, InternVL3.5-8B, and Gemma-3-12B-IT, respectively, exceeding the strongest baseline by up to 68.36 percentage points. JC optimizes continuous adversarial paths based on the observation that effective adversarial inputs can be connected through low-loss regions in image space: path sampling generates diverse jailbreak images, a lightweight proxy classifier guides cross-model transfer, and a distribution-level objective enables universal attacks against different harmful queries. In black-box evaluations, JC's attack success rates on GPT-5.6 and Gemini 3.1 Pro are 24.69 and 27.34 percentage points higher than FigStep, respectively. The authors state that these findings reveal structural vulnerabilities in the safety alignment of multimodal large language models and can be used for more comprehensive safety evaluation.

Related events and news

1 events · 1 news mentions

News that mentions this paper

周报:预训练投毒可植入智能体后门

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Addressing the insufficient diversity, transferability and universality of multimodal jailbreaks, the authors propose the JC framework based on connected low-loss regions.

    Core problem: existing multimodal large model (MLLM) visual jailbreak methods only optimize isolated adversarial examples, which the authors argue leads to shortcomings in three aspects: diversity, transferability, and universality.

    • Setting and importance: MLLMs combine a visual encoder with an LLM backbone, and the visual modality expands the attack surface; the authors cite existing analyses stating that multimodal alignment remains fragile, while such models are being deployed in high-risk domains such as healthcare and autonomous driving.
    • Shortcomings of existing methods (according to the paper): (1) most methods generate only one jailbreak image for a single harmful query, giving a narrow attack surface that is easier to defend against; (2) jailbreak images often fail to transfer to MLLMs other than the model used to generate them; (3) few methods pursue Universal Jailbreak, i.e., a single image that, paired with any text prompt, can make the model answer multiple harmful queries.
    • Core observation/hypothesis: the authors propose that jailbreak effectiveness forms a connected low-loss region in image space rather than isolated points; once this region is found, sampling yields diversity, guided optimization finds a transferable stable subregion, and a distribution-level objective extends to universal behavior—the three capabilities are natural consequences of the same geometric principle.
    • Method proposed in the paper: the Jailbreak Connectivity (JC) framework, which constructs and optimizes a continuous adversarial path using a quadratic Bézier curve, uses two lightweight proxy classifiers to guide transfer, and extends the objective from a single harmful query to a harmful output distribution.
    • Threat model:
      • Goal: manipulate the image input so that the target MLLM answers the harmful question th and produces harmful content yh.
      • Knowledge: two settings, white-box (full access to model parameters and gradients) and black-box (query interaction only, no internal knowledge).
      • Capability: add a perturbation ∥xp−x∥∞≤ϵ to the original image; in the black-box setting, generate images with an open-source base model and transfer them to closed-source models.
      • Victim systems and scope: open-source and commercial MLLMs; the paper restricts to single-turn interaction, and models are tested on isolated prompts with no dialogue history.
  2. What related research exists?

    Related work falls into two categories, text jailbreaks and multimodal jailbreaks; the authors state that the former is limited to the text modality and the latter is mostly isolated-point optimization.

    Text-modality jailbreak attacks

    • GCG [6]—gradient-optimized adversarial suffixes that induce harmful responses across various prompts; FuzzLLM [23] applies fuzzing to black-box adversarial instruction generation; MJP [24] exploits multi-turn dialogue structure to maintain the jailbreak state; ReNeLLM [25] formalizes jailbreak mechanisms through prompt rewriting and scenario nesting.
    • PAIR [26]—introduces iterative refinement, multi-model collaboration and chain-of-thought dynamics to improve attack reliability.
    • Universal suffix attacks [6], [7] and random rewriting, ensemble optimization methods—study the transferability and diversity of text jailbreaks. The paper notes that these techniques act only on the text modality and provide limited understanding of multimodal vulnerabilities arising from the interaction of visual inputs with LLM safety filters.

    Multimodal jailbreak attacks (the paper organizes these into three directions)

    • Prompt-to-Image Injection [9]–[11], [27]—encode harmful instructions into images while keeping text prompts benign, to bypass text-only safety filters.
    • Prompt-to-Image Perturbation [12]–[14], [28], [29]—add small visual perturbations (sometimes jointly optimized with text), exploiting multimodal fusion vulnerabilities.
    • Proxy model transfer attacks [15]–[17], [30], [31]—generate adversarial images on a proxy MLLM and transfer to unseen targets, showing that multimodal jailbreak vulnerabilities often generalize across architectures.

    Baselines and benchmarks compared in this paper

    • Baseline methods: White-box setting compares Adv Example [45], Query Image [42] and Plain Text; transfer setting compares FORCE [46] (reduces reliance on model-specific features via layer-aware regularization and spectral feature rescaling to improve transferability); black-box setting compares FigStep [9] (embeds harmful text directly into images).
    • Benchmarks: MM-SafetyBench [42] (13 prohibited scenarios) and AdvBench [6] (520 harmful behaviors, deduplicated following BAP [43]'s method and mapped to MM-SafetyBench scenarios).

    Authors' positioning: The authors state that existing methods mostly optimize isolated jailbreak images, transferability degrades across architectures, and universal effectiveness is rare; JC unifies diversity, transferability and universality within a region-level framework by exploring connected low-loss regions in image space. The method draws on mode connectivity [32]; a paper footnote explains that existing mode connectivity research focuses on the loss geometry of model parameters or solutions to the same task, whereas this paper studies adversarial connectivity in the input space of multimodal models.

  3. How does the paper address this problem?

    JC uses Bézier path sampling to achieve diversity, dual proxy classifiers to guide transfer, and a distribution-level objective to achieve universal jailbreak.

    Overall idea: change jailbreak generation from point-by-point optimization to the discovery, refinement, and generalization of a connected low-loss adversarial region, corresponding to the three capabilities of diversity, transferability, and universality. Framework name: Jailbreak Connectivity (JC).

    Problem setting

    • MLLM output is modeled as the conditional probability p(y|x,t), where x is the image and t is the text; the attacker minimizes the jailbreak loss Ljail(xp) := −log(p(yh|xp,th)) under the perturbation constraint ∥xp−x∥∞≤ϵ, i.e., maximizes the generation probability of the harmful output yh.
    • Restricted to single-turn interaction; the white-box setting has access to parameters and gradients, and the black-box setting has query access only.

    Region discovery: path sampling achieves diversity

    • Endpoint search: random initialization, run PGD for 1000 iterations to find two distinct local minima x1 and x2 as path endpoints.
    • Path construction: connect the endpoints with a quadratic Bézier curve ϕxc(u)=(1−u)²x1+2u(1−u)xc+u²x2 (u∈[0,1]), with training objective minxc Eu∼ U(0,1) Ljail(ϕxc(u)), constrained so that every point on the path satisfies the perturbation bound; the control point xc is initialized by linear interpolation, and during training u is sampled and xc is updated by backpropagating gradients with PGD.
    • Connectivity verification: on Qwen3-VL-8B-Instruct, Figure 2 shows that linear interpolation reaches a loss of 5.26 near the midpoint, with 12/21 (57.14%) of 21 sampled points succeeding, whereas the optimized Bézier curve has a maximum loss of only 10⁻⁴ along the entire path and 21/21 (100%) success; the authors state this provides empirical evidence for the existence of connected jailbreak solutions.

    Region refinement: proxy models guide transfer

    • Behavior decomposition: the authors decompose a successful jailbreak into two sequential conditions—first bypassing the refusal mechanism, then inducing harmful content generation.
    • Two proxy classifiers: both use CLIP-ViT-Base-Patch32 as the backbone. The Safety Classifier (SC) predicts the probability that the image is regarded as safe by the target model and triggers a response; the Jailbreak Success Predictor (JSP) predicts the probability that, conditional on the model responding, the image induces harmful content. The paper assumes access to a jailbreak image dataset for the target MLLM (constructible with existing methods in white-box or black-box settings) to train these two classifiers.
    • Transfer objective: for the base MLLM, its direct jailbreak loss is used; for each of the remaining n−1 models, a pair of SC/JSP is trained, and the transfer loss Ltransfer is the weighted sum of the two cross-entropy losses for each model; in the total objective, α balances base-model effectiveness and transferability, and β balances SC and JSP. The authors state that a higher α favors attack effectiveness on the base model, and a higher β favors generating "safer" images that facilitate transfer.

    Region generalization: distribution objective achieves universal jailbreak

    • Change the attack objective from a predefined yh to a harmful output distribution Yh, and remove the text input, forcing the adversarial signal to be encoded entirely in the visual modality; the universal loss is Luni(xp)=E_{yh∼Yh}[−log(p(yh|xp))].
    • In practice, Yh is approximated with the 100 harmful sentences from AdvBench; Eq. (5) is the most general form, reducing to the transfer setting when restricted to a single query, and to the diversity setting when α=1.

    Success determination and evaluation method

    • ASR: the proportion of harmful queries that induce a disallowed response, with GPT-6 used as a binary harmfulness judge; to account for generation randomness, 5 independent runs are performed, reporting mean and standard deviation.
    • PPL: following FigStep's approach, response perplexity is computed with Qwen3-8B via token-level negative log-likelihood, where lower means higher likelihood under the evaluating language model; likewise 5 runs are reported as mean and standard deviation.
  4. What experiments did the paper conduct?

    White-box average ASR on the three models is 84.01%–97.95%, and black-box transfer to GPT-5.6/Gemini 3.1 Pro reaches 41.94%/47.05%.

    Experimental setup

    • Models tested: open-source Qwen3-VL-8B-Instruct, InternVL3.5-8B, Gemma-3-12B-IT; commercial GPT-5.6, Gemini 3.1 Pro (black-box).
    • Datasets: MM-SafetyBench (13 disallowed scenarios, defined according to the OpenAI usage policies); AdvBench (520 harmful behaviors, deduplicated by the BAP method and mapped to MM-SafetyBench scenarios); the universal objective is approximated with the 100 harmful sentences from AdvBench.
    • Baselines: white-box comparisons with Plain Text, Adv Example, Query Image; transfer comparison with FORCE; black-box comparison with FigStep. The paper states that all methods are evaluated under the same optimization budget.
    • Metrics and judging: ASR uses GPT-6 as a binary harmfulness judge; PPL is computed with Qwen3-8B; both are reported as mean±standard deviation over 5 independent runs.
    • Proxy classifiers: SC and JSP use CLIP-ViT-Base-Patch32 as the backbone; their training set is the only LLM-generated data in the paper and does not overlap with the evaluation queries.
    • Experiment dates: September 16 to October 20, 2026.

    Main results

    White-box ASR (%, per Table I, average over 13 scenarios, 5 runs):

    Swipe horizontally to read the table

    ModelPlain TextAdv ExampleQuery ImageJC
    Qwen3-VL-8B-Instruct5.86±0.5622.97±0.8824.55±0.9384.01±1.61
    InternVL3.5-8B5.04±0.4826.17±1.0927.83±0.9085.77±1.05
    Gemma-3-12B-IT5.18±0.4824.93±0.9029.59±1.0397.95±0.49
    • The authors state that JC improves average ASR over each model's strongest baseline by 59.46, 57.94, and 68.36 percentage points, respectively, and reaches 100% ASR on all three models in the four scenarios Economic Harm, Health Consultation, Political Lobbying, and Sex.
    • Exceptions: in the Malware Generation scenario, JC on InternVL3.5-8B is only 9.59%, lower than Query Image's 42.33%; Qwen3-VL-8B-Instruct has lower ASR on Government Decision (41.19%) and Fraud (57.54%). The authors believe jailbreak effectiveness still depends on the target model and scenario.
    • PPL (Table II): JC's average PPL is 5.07, 4.57, 5.07, lower than Plain Text's 7.16, 6.79, 6.86, and achieves the lowest PPL in 11 of the 13 scenarios for each model; in the Government Decision scenario, JC's PPL increases on all three models.

    Transfer attack (Table III)

    • Setup: use Qwen3-VL-8B-Instruct as the source model to generate images, transfer to InternVL3.5-8B and Gemma-3-12B-IT, and compare with FORCE.
    • Results: JC's average ASR on the source model is 74.56% (FORCE 72.52%), and on the two transfer targets 70.01% and 78.56% (FORCE 66.14% and 74.62%).
    • Exceptions and authors' interpretation: FORCE is better in some scenarios, including Economic Harm (both transfer targets) and Privacy Violence on Gemma-3-12B-IT; the authors state that JC has stronger average transferability, but effectiveness depends on the target model and scenario.

    Black-box attack (Table IV)

    • Setup: use Qwen3-VL-8B-Instruct as the base model to generate images, without access to the parameters or gradients of GPT-5.6 or Gemini 3.1 Pro, and compare with FigStep.
    • Results: on GPT-5.6, JC's average ASR is 41.94% (FigStep 17.25%, +24.69 percentage points); on Gemini 3.1 Pro, 47.05% (FigStep 19.71%, +27.34 percentage points); JC's ASR is higher than FigStep's on all 13 scenarios for both models.
    • PPL: JC's average PPL is 11.41 (GPT-5.6) and 13.11 (Gemini 3.1 Pro), lower than FigStep's 14.52 and 16.41, and lower on all 13 scenarios.

    Universal attack and ablation

    • Universal attack: on 40 unseen harmful queries, JC on Qwen3-VL-8B-Instruct succeeds on an average of 34/40 over 5 runs, versus 17/40 for Adv Example; the same universal image, transferred to InternVL3.5-8B, is effective on an average of 22 queries.
    • Ablation (Table V, transfer setting, source model Qwen3-VL-8B-Instruct): removing path construction (w/o Path) lowers ASR on the three models by 45.91, 42.33, and 43.52 percentage points, respectively; removing JSP lowers ASR on the two transfer targets by 15.16 and 24.52 percentage points, and removing SC lowers it by 30.69 and 22.94 percentage points; on the source model, removing either classifier slightly increases ASR (the authors explain: the source model does not use proxy guidance, and the freed optimization capacity is used for the source objective). Full JC has the lowest PPL in all configurations (5.84, 3.46, 5.77).
    • Authors' interpretation: the continuous path drives attack effectiveness, the two proxy classifiers drive cross-model transfer, and together the three yield the lowest response perplexity.
  5. What are some points that can be further explored?

    The authors only note that PPL does not directly measure harmfulness; the experiments cover 5 models and two benchmarks, with many details in the supplementary material.

    Limitations and Future Directions Noted by the Authors

    • Limitations of the PPL metric (Conclusion): The authors note that perplexity only reflects fluency under a fixed evaluation model and does not directly measure harmfulness.
    • Implications for defenses rather than a defense scheme (Conclusion): The authors state that MLLM safety alignment may fail at the region level rather than the isolated-input level, and defenses need to reason about structured adversarial regions rather than individual samples; the paper does not propose or evaluate a specific defense.
    • The paper has no dedicated Limitations section and, apart from the above, does not explicitly list other limitations or future work.

    Experimental Coverage

    • Models tested: open-source Qwen3-VL-8B-Instruct, InternVL3.5-8B, Gemma-3-12B-IT, commercial GPT-5.6, Gemini 3.1 Pro, 5 in total; both transfer and black-box experiments use Qwen3-VL-8B-Instruct as the source/base model.
    • Data and scale: MM-SafetyBench 13 scenarios; AdvBench deduplicated and mapped to these scenarios; universal objectives use AdvBench's 100-sentence corpus; universal attacks evaluated on 40 unseen queries.
    • Number of repetitions: Main table results are the mean ± standard deviation of 5 independent runs; ASR judged with GPT-6, PPL with Qwen3-8B.
    • Interaction scope: The paper explicitly limits to single-turn interaction, with models tested on isolated prompts without dialogue history.
    • Details not reported in the main text: Training details of the proxy classifier, fidelity analysis, sensitivity of hyperparameters α/β, specific values of perturbation magnitude ϵ, perceptual quality analysis, and evaluation under deployment-time image transformations (scaling, compression, cropping, blurring) are all stated by the paper to be in the supplementary material, with no numbers given in the main text; the paper does not report agreement between judge decisions and human annotations.
  6. Summarize the main content of the paper

    JC changes multimodal jailbreaking from point-by-point optimization to region exploration; white-box and black-box ASR both exceed baselines, and the authors state it exposes region-level vulnerabilities.

    A multimodal jailbreak attack paper proposing the Jailbreak Connectivity (JC) framework, addressing the problem that "existing methods only optimize isolated adversarial samples".

    • Core hypothesis: The authors argue that MLLM jailbreak effectiveness forms connected low-loss regions in image space; diversity, transferability and universality are natural consequences of exploring the same region rather than independent design objectives.
    • Method highlights: Use PGD to find two jailbreak endpoints and construct a low-loss path with a quadratic Bézier curve, sampling along the path to obtain diverse jailbreak images; use a Safety Classifier and Jailbreak Success Predictor with CLIP-ViT-Base-Patch32 as backbone to guide optimization toward transferable sub-regions; extend the objective to a harmful output distribution and remove text input to achieve single-image universal jailbreaking.
    • Connectivity evidence: On Qwen3-VL-8B-Instruct, all 21 sampling points on the optimized path attack successfully (21/21), while linear interpolation achieves only 12/21; the loss at the path midpoint drops from 5.26 to the order of 10⁻⁴ (Figure 2).
    • White-box results: Average ASR across 13 MM-SafetyBench scenarios is 84.01% (Qwen3-VL-8B-Instruct), 85.77% (InternVL3.5-8B), 97.95% (Gemma-3-12B-IT); the authors state improvements of 59.46–68.36 percentage points over the strongest baseline (Table I).
    • Black-box results: Transferring from Qwen3-VL-8B-Instruct to GPT-5.6 and Gemini 3.1 Pro, average ASR is 41.94% and 47.05%, higher than FigStep's 17.25% and 19.71%, with lower PPL on all 13 scenarios (Table IV). Universal attacks succeed 34/40 on 40 unseen queries (Adv Example 17/40).
    • Exception: In the Malware Generation scenario, JC achieves only 9.59% on InternVL3.5-8B, lower than Query Image's 42.33%; the authors state effectiveness still depends on the target model and scenario.
    • Authors' conclusion: MLLM safety alignment may fail at the region level, and defenses need to target structured adversarial regions; the authors also note that PPL only reflects fluency and does not directly measure harmfulness.

Related papers

Show 3 more papers
Read originalarxiv.org