Jailbreak-AudioBench: A Jailbreak Evaluation Benchmark for Large Audio Language Models

End-to-end LALMs directly read hidden semantics in audio, but such jailbreaks still lack systematic evaluation.

• Scenario: The authors state that voice assistants, customer service, and in-car voice use audio-language processing. End-to-end LALMs, without transcription, directly read raw audio, thus preserving emphasis, speech rate, intonation, pitch, background noise, accent, and emotion. Cascaded models first do ASR then pass to an LLM, and transcription loses this information.
• Existing gaps: The authors state that text and visual jailbreaks have substantial work, but audio jailbreaks of end-to-end LALMs still lack systematic research; there is preliminary work, but systematic investigation is limited.
• Observation: The authors believe that sensitivity to hidden semantics in audio makes end-to-end LALMs more easily bypassed by such edits.
• This paper proposes: Jailbreak-AudioBench, including Toolbox, Dataset, and Benchmark. The authors claim this is the most comprehensive audio jailbreak evaluation to date, and use it for query-based audio edits and a prepended-instruction defense.
• Threat model:
  • Goal: Induce LALMs to answer harmful questions. Success requires Llama Guard 3 to judge harmful and the response to exceed 50 words.
  • Knowledge: The paper does not state whether attackers can read parameters or gradients. The main evaluation does one inference per audio; representation analysis of open-source models reads audio encoder and transformer hidden states.
  • Capability: Convert text jailbreak questions to audio, then use the Toolbox to inject hidden semantics. In the query-based setting, each sample has 32 edit combinations.
  • Victim systems: End-to-end LALMs listed in Section 3.2, including BLSP, SpeechGPT, Qwen2-Audio, SALMONN, VITA-1.5, R1-AQA, MiniCPM-o-2.6, GPT-4o-Audio, Gemini-2.5-Flash.

The authors place this paper alongside text/visual jailbreaks, LALM architectures, and audio capability evaluations, and state that audio-specific threats remain overlooked.

Jailbreak attacks and evaluation data

• Prompt engineering: Related Work states that prompt engineering such as fabricating facts, role-playing, or repeating queries exposes cross-modal vulnerabilities, citing [53; 31; 24; 8; 10; 16]. The introduction also lists imperative wording, role-playing, emoji injection, and distraction redirection that mixes benign and harmful requests; the authors say these methods are simple but can bypass system-level defenses.
• Modality perturbation: On LVLMs, attackers use typographic or visual perturbations to trigger jailbreaks [23; 9]. Another line injects optimized, hard-to-perceive perturbations into inputs, forming adversarial prompts [73; 43] or adversarial images [48; 29; 12; 52; 11].
• Benchmarks and audio work: AdvBench[73], MM-SafetyBench[40], RedTeam-2K[42], SafeBench[23] provide jailbreak prompts. [62; 42; 64; 69] evaluate LVLM jailbreak robustness from both attack and defense sides. On audio, Yang et al.[65] take 350 items from AdvBench and evaluate several end-to-end LALMs [13; 55; 56].

LALM architectures

• Cascaded: Representative designs use Whisper for ASR and GPT-4 for downstream question answering or summarization; GigaSpeech+GPT feeds large-scale ASR output into an LLM. Gao et al.[20] and MERaLiON-AudioLLM connect Whisper to LLaMA and SEA-LION V3 respectively; the authors say these achieve strong performance on multilingual question answering and translation.
• End-to-end: The closed-source representative is GPT-4o. On the open-source side, BLSP uses a lightweight adapter to align a frozen speech encoder with an LLM; SpeechGPT uses discrete units and multi-stage training to unify speech and text; Qwen2-Audio and SALMONN connect audio encoders to LLMs; VITA-1.5 performs real-time speech-visual joint decoding; R1-AQA uses reinforcement learning for audio question answering; MiniCPM-o-2.6 targets low-resource settings with a smaller structure.

Existing LALM benchmarks

• Capability evaluation: AIR-Bench evaluates basic and conversational tasks on speech, natural sounds, and music; AudioBench covers 8 tasks and 26 datasets; MMAU uses 10,000 audio items with question answering for expert-level reasoning; ADU-Bench has 20,000 open-ended multilingual dialogues. FunAudioLLM targets speech understanding and generation; WavLLM uses dual encoders to model semantic and speaker information separately. This section only describes what these benchmarks do.

Baselines and this paper's data

• Comparison: Table 2's Original is the baseline ASR on unedited audio, and edited results are compared with it via relative change.
• Question source: The paper writes N=4700, sourced from 250 items from AdvBench, 1,680 from MM-SafetyBench, 2,000 from RedTeam-2K, and 500 from SafeBench.

The authors state that prior work mainly focused on text and vision, and audio-specific threats are largely overlooked; this paper uses an editing toolbox, explicit/implicit data, and end-to-end LALM evaluation to build an audio jailbreak benchmark. The authors call this the most comprehensive audio jailbreak benchmark and dataset to date.

Jailbreak-AudioBench first converts text jailbreak problems into speech, then injects hidden semantics, using ASR to evaluate end-to-end LALMs; the same toolbox is reused for query-based editing and pre-instruction defenses.

Toolbox

• Conversion and seven editing categories: gTTS for text-to-audio. Edits are Emphasis (Volume ×2/×5/×10), Speed (Rate ×0.5/×1.5), Intonation (Interval +2/+3/+4), Tone (Semitone -8/-4/+4/+8), Background Noise (Crowd/Machine/White Noise), Celebrity Accent (Kanye West/Donald Trump/Lucy Liu), Emotion (Laugh/Scream).
• Implementation: Appendix A states Emphasis uses librosa to amplify segments, typically the first 1.0 second; Speed uses SoX tempo while preserving pitch; Intonation shifts pitch per segment with interval sequences [0, 2, 4, 6], [0, 3, 6, 9], [0, 4, 8, 12]; Tone uses SoX semitone shifts; noise is overlaid at SNR -30dB; accents use Coqui TTS; emotions use Dia-1.6B. Figure 3 uses text annotations and spectrograms to compare before and after editing.
• Number of categories: Table 1 writes 4*Tone+3*Intonation+2*Speed+3*Emphasis+3*Background Noise+3*Celebrity Accent+2*Emotion=20 categories.

Dataset construction

• Classification: Algorithm 1 starts from a base question set, uses GPT-4o plus human annotation to split into Explicit and Implicit by threat level. The paper writes |QEx|=2497, |QIm|=2203, used to evaluate different jailbreak strengths; the criteria for the two categories are not specified. Figure 1 writes Explicit has 16 detail sub-tasks, Implicit has 12; the converted text does not list sub-task names.
• Generation: Both categories go through gTTS to obtain base audio, then each sample undergoes editing operations to obtain Edit(AEx) and Edit(AIm).
• Scale: In Table 1, Explicit has base audio 2497, edits 49940, total 52437; Implicit has 2203, 44060, 46263; Explicit Defense has 2497, 49940, 52437; Explicit Small has base audio 262, 32 categories, edits 8384, total 8646. The main text states the dataset has 157,782 audio samples in total. Appendix Table 3 additionally has Explicit Small (GPT-4o Eval) 262/5240/5502, Implicit Small (GPT-4o Eval) 237/4740/4977, Implicit Defense 2203/44060/46263.

Success determination

• Models and inference: Evaluates BLSP, SpeechGPT, Qwen2-Audio, SALMONN, VITA-1.5, R1-AQA, MiniCPM-o-2.6, GPT-4o-Audio, Gemini-2.5-Flash. Each uses official implementation default hyperparameters, one inference per audio sample.
• ASR: The response must be judged harmful by Llama Guard 3 and exceed 50 words. The authors state the length requirement is to exclude short responses, and state Llama Guard 3 is close to human evaluation; the comparison is in Appendix C.
• Closed-source subset: Due to the high cost of full evaluation, GPT-4o-Audio and Gemini-2.5-Flash are evaluated on smaller versions. Appendix B writes these two small sets have base audio of 262 and 237, plus 20 categories of edits.

Query-based editing and pre-instruction defense

• Query-based: 262 samples are drawn from Explicit, maintaining about one-tenth of harmful content categories, forming Explicit Small. Appendix D writes grid search combines accent/emotion, emphasis, speed, background noise. Table 1's count formula is 2*Speed*2*Emphasis*2*Background Noise*(2*Celebrity Accent+2*Emotion)=32. 262×32=8384. Figure 5 bottom row indicates whether at least one success occurs across 32 queries. The converted text does not list the 32 parameter sets individually.
• Defense: gTTS generates an audio instruction without further editing, requiring the assistant to refuse illegal, harmful, or unethical content; it is prepended to the sample with 1000ms silence inserted, yielding Explicit Defense and Implicit Defense. The authors state this is a lightweight method to strengthen alignment with pre-audio instructions.

Experiment setup

• Tested models: BLSP, SpeechGPT, Qwen2-Audio (Qwen2-Audio-7B in the analysis and Figure 5), SALMONN-7B, SALMONN-13B, VITA-1.5, R1-AQA, MiniCPM-o-2.6, GPT-4o-Audio, Gemini-2.5-Flash. Default hyperparameters.
• Data: Explicit 2497, Implicit 2203 base audios, each with 20 types of edits. Closed-source models use a small set from Appendix B, with base audios 262 and 237. The query format uses Explicit Small, 262×32. Defenses use Explicit Defense and Implicit Defense.
• Metric: ASR. Llama Guard 3 judges harmful and over 50 words. One inference per audio. The paper does not report multiple repetitions.
• Baseline: Unedited Original audio. Table 2 uses red for increases and green for decreases when the absolute change is ≥1%.
• Human comparison: 3 volunteers, evaluating only outputs on Explicit Small original audio across the models listed in Appendix C.
• Representation analysis: audio encoder and certain transformer layers of Qwen2-Audio-7B, MiniCPM-o-2.6, SALMONN-7B, with data from different edits on the Explicit Subtype.

Main results

The table below shows the Original ASR from Table 2, left column for the Explicit side, right column for the Implicit side.

Model	Table 2 left	Table 2 right
BLSP	47.5%	18.25%
Qwen2-Audio	16.8%	6.76%
SALMONN-7B	31.4%	14.3%
SALMONN-13B	31.3%	12.89%
VITA-1.5	3.7%	2.77%
MiniCPM-o-2.6	18.2%	9.03%
GPT-4o-Audio	0.7%	0.8%
Gemini-2.5-Flash	8.1%	5.1%

The body text writes that GPT-4o-Audio and Gemini-2.5-Flash were switched to smaller versions due to cost; the base audios in Appendix B are Explicit Small (GPT-4o Eval) 262 and Implicit Small (GPT-4o Eval) 237. The table notes still write Explicit/Implicit Subtype. SpeechGPT (14.1%/2.45%) and R1-AQA (12.6%/7.17%) are omitted here, but they are also in Table 2.

• The authors state that SALMONN is most susceptible to edits, especially celebrity accent, emphasis, background noise, and emotion. In Table 2, SALMONN-13B's Explicit side shows Machine Noise +28.6%, Semitone +8 +24.1%, Laugh +23.2%, Volume*5 +20.5%.
• The authors group SpeechGPT, Qwen2-Audio, and BLSP together, stating that most edits do not increase ASR. In Table 2, SpeechGPT's Explicit positive changes are all less than 1%; Qwen2-Audio reaches the 1% coloring threshold only for Rate*1.5 at +1.1%. BLSP has Laugh +4.0%, Rate*0.5 +2.8%. The authors state that for VITA-1.5, R1-AQA, MiniCPM-o-2.6, increases are generally within 5%; MiniCPM-o-2.6's Semitone +8 and Kanye West on the Explicit side are +7.9%.
• There are also decreases. SALMONN-7B's Interval+2 on the Explicit side is -27.6%, and SALMONN-13B's Rate*1.5 is -22.9%. The authors state that GPT-4o-Audio only shows increases less than 1.7% on some edits; in Table 2, its Machine Noise on the Implicit side is +1.7%, and Rate*1.5 and Interval+4 on the Explicit side are +1.5%. The authors state that Gemini-2.5-Flash's increases are mainly in speed and intonation; its Interval+3 on the Explicit side is +1.9%.

Query-format audio editing

• What was tested: Each base audio in Explicit Small has 32 edited variants. In Figure 5, columns are samples, the first 32 rows are variants, the second-to-last row is unedited audio, and the bottom row indicates whether at least one of the 32 attempts succeeded. The caption writes green as failure and red as success.
• Results: Figure 5 shows Qwen2-Audio-7B from 13.3% to 48.8%, SALMONN-7B from 31.6% to 85.1%, GPT-4o-Audio from 0.7% to 8.4%, Gemini-2.5-Flash from 8.1% to 49.4%. Appendix Figure 8 shows BLSP from 48.1% to 87.8%, SpeechGPT from 12.2% to 42.4%, VITA-1.5 from 3.4% to 47.7%, MiniCPM-o-2.6 from 16.8% to 65.7%.
• Authors' interpretation: The authors state that combined edits raise ASR, including for closed-source models; they also state that some open-source models claim GPT-4o-level performance on standard metrics, but their resistance to this attack differs from GPT-4o-Audio. The paper does not report query-format ASR for R1-AQA and SALMONN-13B.

Pre-pended audio instruction defense

• What was tested: Prepending a refusal instruction (not further edited) to Explicit and Implicit samples yields Explicit Defense and Implicit Defense. The x-axes of Figure 6 and Figure 9 are BLSP, SpeechGPT, Qwen2-Audio, SALMONN-7B, SALMONN-13B, VITA-1.5, R1-AQA, MiniCPM-o-2.6.
• Results: The authors state that in both figures the defense reduces ASR for all models; the numbers on the bars are the ASR reductions from the defense. The bars for converted text cannot be reliably aligned with model columns, so these reductions are not transcribed item by item here.
• Authors' interpretation: The authors state that the pre-pended audio instruction provides baseline-level protection, but the remaining ASR is still high, and they say this shows the limitation of this strategy and the need for more effective defenses.

Internal representations

• What was tested: Figure 4 shows t-SNE for three models on Explicit Subtype with different edit types. The authors selected Qwen2-Audio-7B, MiniCPM-o-2.6, SALMONN-7B, calling them highly robust, moderately robust, and vulnerable, respectively. The appendix writes that layers include 0, 8, 16, 24, and the last layer.
• Authors' interpretation of the figure: The authors state that at the audio encoder, all three models cluster by edit type rather than by semantics. They state that Qwen2-Audio-7B shifts to semantic clustering by Layer 8, and at Layer 31 the separation among edit types is small; MiniCPM-o-2.6 shifts earlier but incompletely, and Layer 27 is still scattered; SALMONN-7B remains separated by edit type even at Layer 31, and separated from the original audio. The converted text does not show the location of points; all of the above is the authors' description of Figure 4.
• Authors' explanation: The authors use this difference to explain the three models' different responses to audio-edit jailbreaks, and they state that even the more robust model initially encodes edit features separately, so combinations of multiple edits may exceed the normalization in its later layers.

Other ablations and analyses

• Appendix C: Explicit Small original audio, n=262. Llama Guard 3 gives BLSP 126/262 (48.1%), SpeechGPT 32/262 (12.2%), Qwen2-Audio 35/262 (13.3%), SALMONN-7B 83/262 (31.6%), VITA-1.5 9/262 (3.4%), MiniCPM-o-2.6 44/262 (16.8%), GPT-4o-Audio 2/262 (0.7%). Relative differences include volunteer 2's +0.4% for GPT-4o-Audio and volunteer 1's +4.5% for BLSP (138/262). The authors state the largest increase is less than +4.5%, and cases Llama Guard 3 judges as successful are also judged successful by humans, so Llama Guard 3 is adopted.
• Figure 2(b) labels Emphasis 20.5, Speed 16.8, Intonation 5.6, Tone 24.1, Background Noise 28.6, Celebrity Accent 20.1, Emotion 23.2, with axes approximately -30 to +30. The authors state this is the maximum jailbreak threat change caused by hidden semantics across each LALM. In Table 2, SALMONN-13B Explicit Interval+4 is +11.7%, which is larger than the intonation label 5.6 in the figure.
• Appendix C re-plots t-SNE by each edit's parameters; Figures 17 to 24 are UMAP. The authors state UMAP and t-SNE patterns are consistent. Since the positions of points in the figures are not visible, the geometry of the clusters is not described further.

Limitations and future directions noted by the authors

• Judge: Section 6 writes that, following existing jailbreak research, they mainly use Llama Guard 3. After checking 157,782 responses, the authors observed that some responses merely repeat the input; because these contain a few harmful words, Llama Guard 3 misclassifies them as successful attacks. The authors state that current jailbreak evaluation metrics are still imperfect, plan to improve them, and encourage the community to continue research.
• Speech naturalness: Section 6 writes that accurately modeling natural speech with real prosody, speaking rate, and pronunciation variations remains difficult. Currently, TTS audio is used as the raw input, then edited with Toolbox. The authors plan to include natural speech recordings and expand the benchmark to better reflect real scenarios.

Experiment coverage

• The main evaluation reports ASR for 10 end-to-end models under default hyperparameters, with one inference per audio (Section 3.2, Table 2). The paper does not report repetitions beyond one inference.
• The main experiment covers 20 types of edits; the query format uses 32 combinations. Figure 5 reports Qwen2-Audio-7B, SALMONN-7B, GPT-4o-Audio, Gemini-2.5-Flash; appendix Figure 8 reports BLSP, SpeechGPT, VITA-1.5, MiniCPM-o-2.6. The paper does not report query-format ASR for R1-AQA and SALMONN-13B.
• Table 2 for closed-source models corresponds to the 262 and 237 base audios in Appendix B. The x-axes of Figure 6 and Figure 9 are BLSP, SpeechGPT, Qwen2-Audio, SALMONN-7B, SALMONN-13B, VITA-1.5, R1-AQA, MiniCPM-o-2.6.
• The human comparison involved 3 volunteers, Explicit Small original audio (262 items), and the 7 models listed in Appendix C. The representation analysis uses encoder and transformer hidden states from Qwen2-Audio-7B, MiniCPM-o-2.6, SALMONN-7B (Figure 4, Appendix C).
• Section 6 writes that the full run took about 9,216 GPU hours (NVIDIA A40), and the closed-source API cost was $1,000.

Jailbreak-AudioBench uses a toolbox, explicit/implicit audio, and ASR to examine end-to-end LALMs' jailbreak responses to hidden semantic edits.

• The authors are concerned that end-to-end models, without transcription, directly use information such as speech rate, pitch, noise, accent, and emotion; beyond text and visual jailbreaks, such audio edits lack systematic evaluation.
• Questions come from AdvBench, MM-SafetyBench, RedTeam-2K, SafeBench. The paper writes N=4700, split into Explicit 2497 and Implicit 2203; gTTS converts to audio then applies 20 types of edits. Success means Llama Guard 3 judges harmful and over 50 words, with one inference per item.
• Table 2 unedited Explicit ASR includes BLSP 47.5%, SALMONN-7B 31.4%, SALMONN-13B 31.3%, Qwen2-Audio 16.8%, VITA-1.5 3.7%. GPT-4o-Audio 0.7% and Gemini-2.5-Flash 8.1% correspond to the small set in Appendix B. SALMONN-13B's Machine Noise is +28.6%, SALMONN-7B's Interval+2 is -27.6%, both Explicit relative to Original.
• On Explicit Small, 32 combined queries raise ASR from 31.6% to 85.1% (SALMONN-7B), 13.3% to 48.8% (Qwen2-Audio-7B), 8.1% to 49.4% (Gemini-2.5-Flash), 0.7% to 8.4% (GPT-4o-Audio) (Figure 5).
• The authors state that prepended refusal instructions can lower open-source model ASR, but the remaining values are still high. The authors list Llama Guard 3 misjudging repetition, and TTS rather than natural speech, as limitations, and plan to improve metrics and add natural speech recordings.