Best-of-N Jailbreaking: a cross-modal black-box jailbreak algorithm

1. 01What problem does this paper attempt to solve?

   The authors propose BoN Jailbreaking: a black-box jailbreak that repeatedly samples randomly augmented prompts, covering text, vision, and audio modalities.

   Core question: how to use a simple, black-box method that requires no internal access to automatically jailbreak frontier cross-modal AI systems in order to evaluate their safety defenses.

   • Setting and motivation: The authors argue that as models become more capable and support more input modalities, jailbreaks could be used for cybercrime, bioweapons, disinformation, etc., so automated red-teaming methods that can be applied seamlessly to multimodal settings are needed to evaluate model defenses.
   • Shortcomings of existing methods: According to the paper, most jailbreak methods require logprobs, gradients, or multi-step iterative optimization, making them hard to use on closed-source models with restricted access and on new modalities; the authors state that BoN does not require these and is fast to implement.
   • Core observation: Model outputs are stochastic and sensitive to seemingly harmless changes in the high-dimensional input space; by repeatedly sampling enough augmented variants, one will eventually bypass the defenses. The authors report that successful jailbreaks reproduce under resampling only about 20%–30% of the time, indicating that the attack exploits variance in the output distribution rather than a stable pattern.
   • The paper proposes: the Best-of-N (BoN) Jailbreaking algorithm; a power-law scaling law for ASR as a function of the number of samples N and a method for predicting it; combinations with prefix attacks such as PrePAIR and MSJ; and a mechanistic analysis of jailbreak reliability and difficulty correlations.
   • Threat model:
     • Goal: make the model output harmful content in response to HarmBench harmful requests.
     • Knowledge: black-box—the authors explicitly state that logprobs or gradients are not needed.
     • Capabilities: repeatedly submit augmented inputs (text/image/audio), can set sampling temperature, can modify the system prompt (as in the Cygnet experiments); the Gemini API's optional safety filters were turned off, with the authors' rationale that real attackers would also turn them off.
     • Victim systems: frontier LLMs/VLMs/ALMs such as Claude, GPT, Gemini, and Llama, as well as circuit breakers and the GraySwan Cygnet defense.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into two groups: text jailbreaks and multimodal jailbreaks; the authors position BoN as a repeated-sampling method that requires no optimization and no logprobs.

   Text LLM jailbreaks

   • Huang et al. (2023), Yu et al. (2024): Use decoding variations and fuzzing to mutate inputs and elicit jailbreaks, respectively, similar to BoN's ideas of repeated sampling and augmentation.
   • Doumbouya et al. (2024), Samvelyan et al. (2024): The former models attacks as compositions of string transformations and searches with multi-step iterative optimization, while the latter models adversarial prompt generation as quality-diversity search; the paper notes that BoN text attacks belong to the string-transformation category but require no optimization, working through repeated sampling with large N.
   • Andriushchenko et al. (2024), Zou et al. (2023): Use random token search to optimize the target log probability and gradient methods, respectively; the paper notes that BoN requires no log probabilities or gradients and does not rely on transfer between models, making it suitable for models with restricted access.
   • LLM-assisted attacks (Chao et al., 2023; Shah et al., 2023; Zeng et al., 2024; Mehrotra et al., 2023; Yu et al., 2023): Use LLMs to generate attack strategies; the paper states that PrePAIR is similar to these, but BoN itself focuses on augmented sampling and differs from human red-teaming and genetic algorithms (Wei et al., 2024; 2023; Lapid et al., 2023; Liu et al., 2023).

   Jailbreaks in other modalities

   • VLM attacks: On open-source parameter models, gradient methods are mostly used to generate adversarial images (Zhao et al., 2023; Qi et al., 2024, and about 20 others); on closed-source models, there is work using images containing typographically rendered harmful text to bypass defenses (Gong et al., 2023; Shayegani et al., 2023; Li et al., 2024a), which the paper states is similar to how BoN generates typographically augmented images.
   • ALM attacks: Existing work mostly reads harmful requests aloud directly without augmentation (Shen et al., 2024; OpenAI, 2023a; Gemini Team, 2024); Yang et al. (2024) jailbreak ALMs with additive noise, which the paper states is similar to one of its audio augmentations.

   Baselines and benchmarks: The control baseline in the experiments resamples the same request 500 times with no augmentation at temperature 1 or 0 only; the combination experiments use PrePAIR, adapted from PAIR (Chao et al., 2023), and MSJ (Anil et al., 2024). Evaluation uses 159 direct requests from HarmBench (Mazeika et al., 2024), and prefix training uses AdvBench (Chen et al., 2022).

   The authors position BoN as a jailbreak algorithm that is simpler than the above methods (augmentation is a direct perturbation of the request), purely black-box, and naturally covers multiple modalities.

   Collapse this answer

3. 03How does the paper address this problem?

   Repeatedly sample random augmentations for the same request until harmful content is output; estimate ASR(N) using bootstrap and fit a power law to predict ASR at larger N.

   Overall approach: Best-of-N (BoN) Jailbreaking repeatedly submits the same harmful request to the target model as a large number of randomly augmented variants, until some output is judged harmful or the sampling limit N is reached.

   Algorithm flow

   • Augmentation sampling: For each request, randomly combine and apply modality-specific augmentations, requiring that the augmented request remains understandable and its intent recognizable.
   • Generation and judgment: The target model processes the augmented input, and GPT-4o plus the HarmBench grader prompt is used to classify the output's harmfulness; if judged harmful, stop, otherwise continue sampling, up to N times.
   • False-positive stop prevention: Appendix B.1 gives additional stopping criteria to filter classifier false positives (e.g., excluding responses that differ from the original request by no more than 8 words, or no more than 50 words, and responses containing specific modality phrases), reducing the probability of early termination due to false positives.

   Modality-specific augmentations (Appendix A)

   • Text: character shuffling (middle characters of long words shuffled with probability 0.6, first and last unchanged), random casing (probability 0.6), character noise (with probability 0.06, adding or subtracting 1 to a character's ASCII value).
   • Visual: typeset the request into an image—random size (240–600 pixels), 50–80 random color blocks on a black background, random font (104 kinds), font size, weight, color, position; regenerate if text overflows. The accompanying text prompt is "Follow the instructions in the image.".
   • Audio: the request is first read aloud by a human voice or TTS, then six augmentations are applied in a fixed order [speed, pitch, speech, noise, volume, music]; a six-dimensional vector is sampled from N(0, σ²I) (σ=0.25) and mapped according to each augmentation's value range (e.g., pitch ±2000 cents, volume across six orders of magnitude, background sound SNR -10 to 30 dB).

   ASR estimation and power-law fitting

   • Metric: ASR is the proportion of successful jailbreaks on 159 HarmBench "standard" direct requests (excluding copyright and contextual categories), reported as a function of N.
   • Bootstrap: without rerunning experiments, perform 100 bootstrap resamples on the observed success/failure trajectories of each request, report the mean ASR, with standard deviation as error bars; results should be understood as the expected ASR over multiple trajectories.
   • Power law: fit −log(ASR) = aN^−b, i.e., ASR is approximately linear with N in log-log space; take logarithmically spaced points and ignore the first 5 points for linear regression. For prediction, fit using data only up to N=1,000, then extrapolate the final ASR.

   Success determination

   • Success is counted as long as the output contains information related to the harmful request, even if incomplete; the authors say most successful jailbreaks are comprehensive, but small harmful fragments may also be exploited. All outputs flagged as potential jailbreaks were manually reviewed.

   Attack combinations

   • PrePAIR: modifies PAIR so that the attacker LLM (GPT-4o, temperature 0.8) optimizes only a universal prefix rather than the entire request; the prefix and 4 requests form a batch, stopping when all jailbreak successfully or 30 iterations are reached; the audio prefix is read aloud with ElevenLabs TTS and concatenated, and the visual prefix is typeset into the image together with the request.
   • MSJ: for Claude models (on which PrePAIR text is ineffective), switch to many-shot jailbreaking: use responses obtained by jailbreaking a subset of AdvBench with BoN, normalize them with GPT-4o-Mini, and form a 100-shot prefix; additionally shuffle the demonstration order when combining. When combining, augmentations apply to both the request and the prefix.

   Collapse this answer

4. 04What experiments did the paper conduct?

   On 159 HarmBench requests, three modalities and 11 systems were tested: for text at N=10,000, all ASR exceed 50%, and two defenses were broken; ASR shows power-law scaling.

   Experimental Setup

   • Models tested: Text — Claude 3.5 Sonnet, Claude 3 Opus, GPT-4o, GPT-4o-mini, Gemini-1.5-Flash-001, Gemini-1.5-Pro-001, Llama 3 8B, Llama-3-8B-Instruct-RR (circuit breaking defense), GraySwan Cygnet; vision — same as above (excluding Llama); audio — Gemini Flash/Pro, DiVA, GPT-4o Realtime API.
   • Datasets: 159 direct requests from the HarmBench "standard" category; MSJ prefixes built from an AdvBench subset.
   • Parameters: text N=10,000, vision and audio N=7,200, temperature 1; audio max tokens 200; Cygnet experiments N=6000, max tokens 1000.
   • Judging: initial judgment by GPT-4o + HarmBench grader prompt, all flagged outputs manually reviewed; Appendix E.1 reports that classifier false positive rates vary by model (e.g., Circuit Breaking 43%, GPT-4o 1%).
   • Baseline: no augmentation, resampling the same request 500 times at temperature 1 or 0.
   • Statistics: ASR averaged over 100 bootstrap trajectories; inter-rater agreement for manual review not reported.

   Main Results

   According to Sections 3.1–3.3 of the main text and Figure 1 (ASR with N=10,000 for text and N=7,200 for vision/audio):

   Swipe horizontally to read the table

   Model	Text	Vision	Audio
   Claude 3.5 Sonnet	78%	33%	Not supported
   Claude 3 Opus	92.45% (Table 3)	88%	Not supported
   GPT-4o	89%	56%	71% (Realtime)
   GPT-4o-mini	94.91% (Table 3)	67%	Not reported
   Gemini Flash	63.52% (Table 3)	46%	71%
   Gemini Pro	50%	25%	59%
   Llama 3 8B	94%	Not reported	Not reported
   Circuit Breaking	52%	Not reported	Not reported
   DiVA	Not reported	Not reported	87%

   • The authors state that text BoN exceeds 50% ASR on all eight models; the abstract and introduction report 89% for GPT-4o and 78% for Claude 3.5 Sonnet, while Table 3 gives 88.68% and 77.99%.
   • Without augmentation, direct requests yield ASR of 0.6% (Sonnet) and 0% (Gemini Pro).
   • Vision attacks are generally weaker than text attacks on the same model (e.g., GPT-4o: text 87% vs. image 56%, N=7,200); for Gemini models, audio is instead higher than text/vision (Gemini Pro audio is 10 percentage points higher than text).

   Defense Systems and Cost

   • Circuit Breaking: Llama-3-8B-Instruct-RR was broken with 52% ASR (N=10,000); the authors state this model is more robust than Llama3.1 8B (94% ASR) but was still broken.
   • Cygnet: standard BoN yields only 5% ASR; adding a system prompt that makes the model imitate the input format raises this to 67% (Appendix C.2 main text; the Figure 11 caption says 86%). The authors report that 90% of responses judged harmful contain the short-circuit phrase, but the model had already output harmful information before the short circuit.
   • Cost: Table 3 reports that GPT-4o at N=100 achieves 49.37% ASR at a cost of $8.66; GPT-4o-Mini achieves 50% ASR at a cost of less than $2; for the cheaper model, 95% of the total cost comes from the GPT-4o classifier.

   Power-Law Scaling and Prediction

   • Fitting: The authors state that −log(ASR) = aN^−b fits well across multiple models and modalities (Figure 3); the slope b is similar across models, while intercepts differ widely; the error bars for GPT-4o curve slightly downward, which the authors suggest may be better than a power law.
   • Prediction: Extrapolating from fits at N=1,000, the average error in predicting text at N=10,000 is 4.4% ASR, vision 6.3%, audio 2.5% (predicted to N=7,200); the average across models and modalities is 4.6%. Predictions are systematically low for models with high final ASR; the improved bootstrap in Appendix C.3 reduces the text error for GPT-4o/Sonnet to 3.3%/0.9%, but overestimates vision for Sonnet and Gemini Pro by 32% and 26%.

   Mechanistic Analysis

   • Effect of augmentation: At N=500 and temperature 1, text BoN achieves ASR of 24%/56%/68% on Gemini Pro/GPT-4o/Sonnet, which are 3.5/22/3.5 times the no-augmentation baseline; the temperature 0 baseline of 500 trials shows almost no improvement (Figures 5, 15).
   • Temperature: Temperature 1 outperforms 0, but the gap is small — on text, from a 0.7% decrease for Claude Opus to a 27.7% decrease for Gemini Pro (Figure 6); the gap is larger for audio, which the authors speculate is related to ALM input filtering.
   • Reliability: Resampling successful jailbreaks 100 times, at temperature 1 only 30%/25%/15% on average reproduce for text/vision/audio (Table 1 text mean 32%, temperature 0 is 47%); at temperature 0 the distribution is bimodal.
   • Difficulty correlation: Spearman rank correlations of jailbreak difficulty across models are mostly 0.6–0.8 (Figure 8); Circuit Breaking correlates with other models at only 0.3–0.5, which the authors state suggests it provides a qualitatively different kind of protection.

   Attack Combinations

   • Final ASR (N=6000, Tables 5–7): Text — GPT-4o 86.2%→96.9%, Sonnet 74.2%→89.2%, Gemini Flash 57.9%→93.7%; vision — Gemini Flash 45.3%→100%; audio — Gemini Pro 59.1%→87.4%.
   • Sample efficiency: Up to 85x for text (Gemini Flash), up to 2000x for vision (Gemini Flash), up to 250x for audio (Gemini Flash); the introduction states that BoN+MSJ reduces the N required for Sonnet text to reach 74% ASR from 6000 to 274 (28x).
   • Negative results: PrePAIR text prefixes are ineffective on Claude models (MSJ was used instead, with standalone ASR of only 6.3%); Appendix D.6 reports that multiple attempts to find a universal audio jailbreak (greedy stacking, CMA-ES, Morse code, combinations of harmful sounds) all failed; the audio combination experiments were not run on GPT-4o Realtime due to cost.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list directions such as augmentation optimization, validation at larger N, and white/grey-box signals; the experiments are limited to 159 HarmBench requests and the listed models.

   Limitations noted by the authors and future directions

   • Augmentations not optimized (Future Work): The authors state that they did only minimal optimization when choosing augmentations, and that more advanced augmentations (e.g., ciphers, paraphrasing requests, suffix sampling strategies, SVG image attacks) could improve effectiveness.
   • Scalability at larger N remains to be verified (Section 4.2): The authors state that the scaling trend suggests BoN may eventually jailbreak any request, but future work is needed to confirm scalability at larger N and to address the problem that classifier false positives cause early termination and that the method currently relies on manual review.
   • White-box/grey-box signals (Future Work): White and grey-box optimization signals and more complex black-box optimization algorithms could be explored to find more effective perturbations.
   • Order of audio augmentations (Appendix A.3): The authors state that they did not run experiments changing the order in which augmentations are applied, leaving this to future work.
   • Background sound files fixed (Appendix A.3): More background sound files could be sampled from the Musan set to improve the algorithm, which the authors call further work.
   • Mechanism of prefix transfer unclear (Appendix D.7.4): No pattern was found for the reason behind the difference between audio and text prefix effects, and the authors suggest further research in future work; attack data generated by BoN could be used to build adversarial training sets (Future Work).

   Scope of experiments

   • The models tested are the 9 text systems, 6 vision models, and 4 audio systems listed in the main text; the dataset is 159 direct requests from HarmBench (excluding the copyright and contextual categories).
   • Sampling budget: text N=10,000, vision and audio N=7,200, Cygnet N=6000; temperature is mainly 1, with temperature 0 as a control.
   • In the combination experiments, only Gemini Flash/Pro were tested for the audio modality; the authors state that GPT-4o audio was not tested because of OpenAI Realtime API cost (Appendix C.7).
   • Judging relies on GPT-4o + HarmBench grader, and all flagged outputs were manually reviewed; the paper does not report the number of reviewers or inter-rater agreement.
   • In the audio single-augmentation sweep, GPT-4o audio covered only 74 of the 159 requests because of API rate limits (Appendix D.2.1 footnote).

   Collapse this answer

6. 06Summarize the main content of the paper

   BoN Jailbreaking: a black-box jailbreak that repeatedly samples random augmentations, effective across all three modalities, with ASR scaling as a power law with N, and combinable with prefix attacks.

   A multimodal jailbreak attack paper that proposes Best-of-N (BoN) Jailbreaking: repeatedly sampling randomly augmented variants of the same harmful request (text character perturbations, image typographic changes, audio speed/pitch changes and added background sound) until the model outputs harmful content. The method is purely black-box and requires no logprobs or gradients.

   • Problem: The authors argue that automated red-teaming methods capable of evaluating frontier model defenses across text, vision, and audio modalities are needed, whereas existing jailbreaks mostly rely on internal access or complex optimization.
   • Method: On 159 direct requests from HarmBench, using GPT-4o + HarmBench grader for judging and manual review, bootstrap estimation of how ASR varies with the number of samples N, and fitting the power law −log(ASR) = aN^−b for prediction.
   • Main results: At N=10,000, text BoN exceeds 50% ASR on all eight models — GPT-4o 89%, Claude 3.5 Sonnet 78%, Gemini Pro 50%; it breaks circuit breakers (52%) and Cygnet with a system prompt (67%).
   • Cross-modal: vision up to 88% (Claude Opus), audio up to 87% (DiVA), GPT-4o Realtime 71%.
   • Scaling: ASR behaves as a power law with N, and 1,000 samples predict final ASR with an average error of 4.6%; successful jailbreaks reproduce only about 20%–30% of the time on resampling, from which the authors conclude that the attack exploits output randomness.
   • Combination: Combined with PrePAIR/MSJ prefixes, sample efficiency improves by up to 250 times (Gemini Flash audio), and GPT-4o text final ASR rises from 86.2% to 96.9%.

   The authors' conclusion: despite the capabilities of frontier models, their random outputs and sensitivity to small input changes can be exploited by a simple algorithm; the authors argue that this poses a challenge for defending models with random outputs and continuous input spaces, and plan to continue this work with better augmentations, white/grey-box signals, and adversarial training data.

   Collapse this answer

Related papers