ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs

1. 01What problem does this paper attempt to solve?

   The authors argue that safety alignment that interprets only semantics has a vulnerability, and propose the ASCII art recognition benchmark VITC and the jailbreak attack ArtPrompt.

   Core problem: the assumption that safety alignment corpora are interpreted only semantically does not hold, and whether the resulting vulnerability can be exploited by malicious users to jailbreak aligned LLMs.

   • Problem scenario: The authors point out that in real-world scenarios such as forums, users often use ASCII art (images formed from text characters) to convey image information; such content conveys meaning through character arrangement rather than semantics, and existing alignment techniques do not cover it.
   • Shortcomings of existing methods: The authors state that alignment methods such as data filtering, supervised fine-tuning, RLHF, and red teaming all focus only on the semantics of training corpora, ignoring that corpora can have multiple interpretations.
   • Core observation: After constructing the VITC benchmark, the authors found that five LLMs—GPT-3.5, GPT-4, Gemini, Claude, and Llama2—all struggle to recognize ASCII art inputs, and as the number of characters increases, recognition accuracy drops to near zero.
   • Paper outputs: Proposes the Vision-in-Text Challenge (VITC) benchmark and the jailbreak attack ArtPrompt—replacing words in prompts that easily trigger refusal with ASCII art to bypass safety measures.
   • Threat model:
     • Goal: Induce unsafe behavior from aligned victim LLMs (jailbreak).
     • Knowledge: Requires only black-box access, which the authors call a practical attack.
     • Capability: Controls the prompts sent to the victim model, replacing sensitive words with an ASCII art generator; multiple cloaked prompts can be sent in parallel.
     • Victim systems: Five safety-aligned LLMs (GPT-3.5, GPT-4, Claude, Gemini, Llama2).

   Collapse this answer

2. 02What related research exists?

   Related work falls into three categories: jailbreak attacks, defenses, and ASCII art; this paper uses five attacks as baselines and compares them on AdvBench and HEx-PHI.

   Jailbreak attacks

   • Zou et al. (2023), Jones et al. (2023): gradient-based methods search for inputs that trigger inappropriate outputs; Liu et al. (2023) (AutoDAN) uses a hierarchical genetic algorithm to automatically generate jailbreak prompts.
   • Chao et al. (2023) (PAIR): uses a pretrained LLM to iteratively generate adversarial prompts; Mozes et al. (2023), Kang et al. (2023) exploit instruction-following behavior to break safety; Li et al. (2023a) (DeepInception) exploits anthropomorphic capabilities to construct nested scenarios.
   • Manual and context-based: Deng et al. (2023), Yu et al. (2023) manually construct jailbreak prompts; Wei et al. (2023b), Wang et al. (2023b) use in-context demonstrations.

   Defenses

   • Detection-based: Jain et al. (2023) detect using input perplexity; Helbling et al. (2023) have the LLM self-check its outputs; SmoothLLM (Robey et al., 2023) randomly perturbs inputs and aggregates outputs.
   • Mitigation-based: paraphrase and retokenization in Jain et al. (2023); replayable autoregressive reasoning in Li et al. (2023b); safety-aware decoding in Xu et al. (2024); other work enhances safety awareness via in-context demonstrations (Wei et al., 2023b; Xie et al., 2023; Zhang et al., 2023b).

   ASCII art text

   • Wagner (2023): ASCII art can use printable characters to generate visual content and enrich textual expression; this paper focuses on ASCII art in the form of richly formatted text.

   Baselines and benchmarks: the comparison baselines are five attacks—Direct Instruction, GCG, AutoDAN, PAIR, and DeepInception—and the benchmarks are AdvBench and HEx-PHI; the defense evaluation uses Perplexity, Paraphrase, and Retokenization.

   Author positioning: compared with optimization-based attacks, ArtPrompt masks a limited number of words and requires no iterative search; compared with manual prompts, it can be automated; the authors also claim that cloaked prompts are human-readable and more covert than attacks that manipulate tokens.

   Collapse this answer

3. 03How does the paper address this problem?

   First use the VITC benchmark to quantify LLMs' ASCII art recognition ability, then use the two-step attack ArtPrompt to replace sensitive words with ASCII art.

   The overall approach has two parts: construct the Vision-in-Text Challenge (VITC) benchmark to quantify LLMs' ability to recognize ASCII art, then, based on the observation of poor recognition ability, design the two-step jailbreak attack ArtPrompt.

   VITC Benchmark Design

   • Datasets: VITC-S contains 8424 samples in 36 classes; each sample is ASCII art of a single character (0–9 or A–Z in upper and lower case), generated with the Python art library and manually screened to 234 fonts; VITC-L contains 8000 samples in 800 classes with 10 fonts; each sample is a character sequence of length 2–4 (lengths 2/3/4 account for 80%/15%/5% respectively), and the label is the concatenation of the individual character labels (Table 1).
   • Task: The LLM predicts the sample label under the task description prompt x0; a prediction matching the ground truth counts as successful recognition.
   • Metrics: Acc is the proportion of correctly predicted samples; AM R is the average proportion of matched characters in the label relative to the label length, used for partial matching in VITC-L (e.g., when predicting a1c against ground truth a7c, AM R = 66.66%).
   • Evaluation strategies: three query strategies: zero-shot, few-shot ICL, and CoT.

   Two-Step Design of ArtPrompt

   • Step I Word masking: Mask words in the attacker's prompt that may trigger refusal; in the worst case, N words generate N masked prompts; the authors state that compared with the discrete infinite search space of optimization-based attacks, the words that need masking are finite, making it more efficient.
   • Step II Generate cloaked prompt: Use an ASCII art generator to replace the masked words with ASCII art, splice them back into the masked prompt to obtain the cloaked prompt, and send it to the victim LLM; N masked prompts can generate N cloaked prompts sent in parallel to reduce latency.
   • Two configurations: Top 1 uses only the font with the highest ASR; Ensemble does not restrict the font.

   Font Selection

   • Using the Acc of each font on VITC-S, fonts are divided into a head-set with high Acc on all models (e.g., alphabet, cards, letters, keyboard, puzzle) and a tail-set with low Acc on all models (e.g., block, roman, xchartri, hollywood, ghoulish); additionally, GPT-4 is used to generate a font Gen, which is added to the head-set (Appendix A.3, Figures 6, 7).

   Success Determination

   • HPR: The proportion of queries that are not refused, determined with Dict-Judge (the setting of Zou et al., 2023).
   • HS: GPT-4 scores responses 1–5 according to GPT-Judge (Qi et al., 2023), where 1 is harmless and 5 is extremely harmful.
   • ASR: The proportion of responses with HS = 5; the authors state that only extremely harmful responses are counted to reduce subjectivity.

   Collapse this answer

4. 04What experiments did the paper conduct?

   On VITC, the five models achieve a maximum recognition Acc of 25.19%; on AdvBench, ArtPrompt Ensemble achieves an average ASR of 52%, and it remains 35%–52% under the three defenses.

   Experimental Setup

   • Models tested: GPT-3.5 (0301/0613/1106 for VITC, 0613 for attacks), GPT-4 (0314/0613/1106 for VITC, 0613 for attacks), Gemini (Pro), Claude (v2), Llama2 (Chat-7B/13B/70B for VITC, Chat-7B for attacks).
   • Datasets: 50 representative non-duplicate harmful instructions refined from AdvBench; HEx-PHI covers 11 prohibited categories with 30 instructions each, and 10 per category are sampled in the experiments.
   • Baselines: DI, GCG, AutoDAN, PAIR, DeepInception, all with default settings; GCG/AutoDAN use transfer settings for black-box models (GCG optimizes a universal suffix against Vicuna-7B, AutoDAN uses prompts optimized on Llama2).
   • Metrics and judging: HPR uses Dict-Judge; HS is scored 1–5 by GPT-4 according to GPT-Judge; ASR is the proportion of HS = 5.
   • Defenses: PPL-Pass (perplexity computed with GPT-2, threshold T = 175.57), Paraphrase (rewriting with GPT-3.5), Retokenization (BPE-dropout dropping 20% of merges).
   • Generation configuration: temperature = 0 for closed-source models; temperature = 0.6, top-p = 0.9 for Llama2.

   Main Results

   VITC recognition results (zero-shot, according to Table 2):

   Swipe horizontally to read the table

   Model	VITC-S Acc	VITC-L Acc	VITC-L AMR
   GPT-3.5 (0301)	10.64%	0.01%	54.39%
   GPT-3.5 (0613)	13.50%	0.10%	53.16%
   GPT-4 (0613)	25.19%	3.26%	19.64%
   Gemini (Pro)	13.00%	0.31%	13.90%
   Claude (v2)	11.16%	0.25%	22.04%
   Llama2 (Chat-7B)	1.01%	0.44%	3.66%
   Llama2 (Chat-70B)	10.04%	0.83%	5.89%

   AdvBench attack results (according to Table 3):

   Swipe horizontally to read the table

   Attack	GPT-3.5 ASR	GPT-4 ASR	Claude ASR	Gemini ASR	Llama2 ASR	Average ASR
   DI	0%	0%	0%	6%	0%	1%
   GCG	54%	10%	4%	46%	18%	26%
   AutoDAN	18%	10%	0%	8%	36%	14%
   PAIR	38%	30%	0%	50%	22%	28%
   DeepInception	16%	0%	0%	78%	14%	22%
   ArtPrompt (Top 1)	72%	16%	20%	60%	14%	36%
   ArtPrompt (Ensemble)	78%	32%	52%	76%	20%	52%

   • The authors state that ArtPrompt has the highest ASR on GPT-3.5, GPT-4, Claude, and Gemini, and that its average HPR of 84%, HS of 3.6, and ASR of 52% are all the highest.
   • Exception: On Llama2, the ASR of AutoDAN (36%) and PAIR (22%) is higher than that of ArtPrompt Ensemble (20%); the authors point out that these two fail to generalize their effectiveness to other models.
   • The authors state that all models perform poorly on VITC, and that for Llama2 recognition Acc rises from 1.01% to 10.04% from 7B to 70B.

   HEx-PHI and Efficiency

   • HEx-PHI: Figure 3 gives the HS of ArtPrompt across the 11 prohibited categories; the authors state that all victim models are induced to produce unsafe behavior in every category; when instructions are sent directly, most models behave safely, but after rewriting with ArtPrompt, even GPT-4 outputs unsafe content.
   • Efficiency: Figure 4 gives ASR and the average number of optimization iterations; the authors state that ArtPrompt requires only 1 iteration and has the highest ASR because cloaked prompts can be sent in parallel; optimization-based attacks such as GCG cannot parallelize iterations.

   Defense Evaluation

   • PPL-Pass and Retokenization: The authors state that ArtPrompt bypasses these two defenses on all victim models; Ensemble + Retokenization has an average ASR of 52% (Table 4), and the authors speculate that the spaces introduced by Retokenization form a new font, which instead lowers the probability of triggering safety measures.
   • Paraphrase: The authors call it the most effective defense against ArtPrompt, with an average ASR of 39% and HS of 3.18 under Ensemble (Table 4), because rewriting may disrupt the arrangement of the ASCII art; however, the authors state it is still insufficient to mitigate the attack.

   Fine-tuning Experiment

   • Setup: Fine-tune GPT-3.5 with 60% of VITC-S and test on 40%; recognition Acc rises from 10.26% to 71.54%.
   • Attack effect: In the fine-tuned model's responses, the proportions of HS = 5/4/2 are 64%/10%/12%, versus 72%/8%/6% for the original model (Figure 5 gives the HS distribution under the Top-1 font); the authors believe that providing corpora that should not be interpreted only semantically can mitigate the vulnerability.

   Other Ablations and Analysis

   • Font: After switching the Ensemble to tail-set fonts, the average ASR drops from 52% to 38% (Table 5).
   • Arrangement: After changing the ASCII art to vertical arrangement, the Ensemble average ASR drops from 52% to 31% (Table 5); the authors speculate that vertical arrangement lowers recognition accuracy, making the model uncertain about the input.
   • Number of masked words: 1 word is masked by default; 2-Word Mask has an average ASR of 32%, and Mask Ensemble 48% (Table 7, Top-1 font); ASR slightly decreases for GPT-3.5, Gemini, and Llama2, and increases for GPT-4 and Claude.
   • Prompting strategies: ICL and CoT yield only marginal improvements on the recognition task, and increasing the number of shots does not necessarily improve performance (Figure 8).

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors note that multimodal models were not evaluated and that fine-tuning-based mitigation is left for future work; the experiments cover five LLMs, two benchmarks, and three defenses.

   Limitations and Future Directions Noted by the Authors

   • Multimodal models not evaluated (Section 7 Limitations): The performance of ArtPrompt on multimodal language models requires further study; the authors assume it remains effective, reasoning that cloaked prompts are still in text format and will confuse the model.
   • Mitigation direction left as future work (Section 5.2): The authors believe that using corpora that should not be interpreted only semantically for pretraining and safety alignment may further mitigate the vulnerability, but defer further exploration to future work.
   • More advanced defenses needed (Section 5.2 experimental observations): The authors state that ArtPrompt bypasses existing defenses, highlighting an urgent need for more advanced defense mechanisms.

   Scope of Experimental Coverage

   • The models tested are five in total: GPT-3.5, GPT-4, Gemini, Claude, and Llama2 (attack experiments use GPT-3.5 0613, GPT-4 0613, Claude v2, Gemini Pro, and Llama2 Chat-7B).
   • Attack evaluation uses 50 instructions from AdvBench and 10 per category from HEx-PHI (11 categories); defense evaluation includes three defenses: PPL-Pass, Paraphrase, and Retokenization.
   • Harmfulness is judged by GPT-4 scoring 1–5 according to GPT-Judge, and ASR counts only HS = 5; the paper does not report agreement between the judge and human annotators.
   • The fine-tuning experiment is conducted only on GPT-3.5 with VITC-S (60% training, 40% testing).
   • The paper does not report the number of queries or time cost of each attack on the victim models; the efficiency evidence is the average number of optimization iterations in Figure 4.

   Collapse this answer

6. 06Summarize the main content of the paper

   The VITC benchmark shows that the five models struggle to recognize ASCII art; ArtPrompt uses this to jailbreak, achieving an average ASR of 52% on AdvBench and bypassing three defenses.

   A jailbreak attack paper: the authors argue that the assumption that safety alignment interprets only semantics does not hold, and around this vulnerability they provide a benchmark and an attack.

   • Problem to solve: Inputs such as ASCII art that convey meaning through character arrangement cannot be interpreted semantically; the authors aim to verify whether LLMs therefore have an exploitable security vulnerability.
   • Method highlights: Construct the VITC benchmark (VITC-S 8424 samples, VITC-L 8000 samples) to measure recognition ability; ArtPrompt has two steps—mask words that easily trigger refusal, then replace them with ASCII art to generate a cloaked prompt; requires only black-box access.
   • Recognition results: The five LLMs achieve a maximum Acc of 25.19% on VITC-S (GPT-4 0613) and a maximum of 3.26% on VITC-L (Table 2).
   • Attack results: On AdvBench, ArtPrompt Ensemble achieves an average ASR of 52%, higher than the five baselines; 78% on GPT-3.5 and 52% on Claude (Table 3); the exception is that AutoDAN and PAIR are higher on Llama2.
   • Defenses and mitigation: Under the three defenses PPL-Pass, Paraphrase, and Retokenization, the average ASR remains 35%–52% (Table 4); after fine-tuning GPT-3.5 with VITC-S, recognition Acc rises to 71.54%, and the proportion of responses with HS = 5 drops from 72% to 64%.
   • Authors' conclusion: The authors state that interpreting corpora only semantically creates a vulnerability exploitable by ArtPrompt and that more advanced defenses are needed; effectiveness on multimodal models and using non-semantic corpora for alignment are left as future work.

   Collapse this answer

Related papers