Toward understanding the transferability of adversarial suffixes in large language models

1. 01What problem does this paper attempt to solve?

   Cross-prompt and cross-model transfer of GCG suffixes lacks fine-grained explanation; the authors use refusal direction geometry to analyze it.

   Why adversarial suffixes obtained through discrete optimization transfer to prompts and models that were not optimized remains lacking a fine-grained explanation.

   • Scenario: The authors state that short, meaningless suffixes appended to prompts requesting harmful content elicit prohibited content, and often succeed on prompts and models not involved in optimization. They compare this to the transferability of image adversarial examples and call it a key phenomenon in LLM jailbreaks.
   • Gap: The authors state that empirical evidence for suffix attack transfer exists, but a fine-grained understanding of when, why, and to what extent transfer occurs is lacking. Jailbreak strategies are diverse, and the authors believe this makes identifying transfer principles more difficult.
   • Observation: The authors aim to answer three types of questions: why some prompts are more easily broken by suffixes, which suffix properties enable transfer, and what internal mechanisms govern success. The corresponding hypotheses are: prompts more aligned with the refusal direction transfer less easily; suffixes that push away from the refusal direction in an antiparallel manner transfer more easily; and offsets orthogonal to that direction play a secondary, model-dependent role. Prompt semantic similarity is only weakly correlated.
   • Approach: They analyze both within-model transfer (same model, different prompts) and cross-model transfer (different models, prompts may be the same or different). They optimize 10,000 GCG suffixes per model, quantify these geometric quantities with statistical models, and intervene by rewriting prompts and modifying the GCG loss. The authors hope the analysis can inform future defense design.

   Collapse this answer

2. 02What related research exists?

   Related work covers adversarial example transfer, jailbreak transfer, and refusal directions; the authors position this paper as a statistical and intervention analysis.

   Adversarial example transferability

   • Empirical observation: the paper states that over the past decade, adversarial attack transferability has appeared across multiple data modalities, architectures, and training schemes (Carlini & Wagner, 2018; Goodfellow et al., 2015; Neekhara et al., 2019; Ren et al., 2019; Taori et al., 2019).
   • Conditions and other factors: Tramèr et al. (2017) identified distributional conditions that cause transfer in linear and quadratic models. Demontis et al. (2019) argue that other factors such as model complexity and gradient similarity also affect transferability.
   • Non-robust features: Ilyas et al. (2019) found that different models tend to learn similar non-robust features and are therefore susceptible to transfer attacks.

   Jailbreak transferability

   • Multiple strategies: Jain et al. (2023), Robey et al. (2025), and Zou et al. (2024) are cited to show that multiple jailbreak strategies transfer across LLMs and have renewed attention to model safety. The authors state this diversity makes identifying transfer principles more difficult. The introduction also uses Wei et al. (2023), Andriushchenko et al. (2025), and Chao et al. (2025) to support the empirical conclusion that jailbreaks often transfer across models.
   • Suffix family: Jones et al. (2023), Liu et al. (2024), Zhu et al. (2024), and in the introduction Wallace et al. (2019), Geisler et al. (2024), and Zou et al. (2023). The authors state that attacks of this type are structurally similar, making it easy to separate the effects of prompts and suffixes, so they only analyze the most commonly used GCG (Zou et al., 2023).

   Mechanistic analysis of model safety

   • Refusal vector: Arditi et al. (2024) identified the refusal vector: subtracting it from the activations of harmful prompts reduces refusal, and adding it to harmless prompts triggers refusal. The refusal direction in this paper follows that definition.
   • Jailbreaks and representations: the paper states that subsequent work found that different jailbreak strategies alter the model's internal representation of harmfulness in different ways, often making harmful prompts look more like benign prompts (Ball et al., 2026; Jain et al., 2024; Lin et al., 2024).

   Data and algorithms used in this paper

   • JailbreakBench (Chao et al., 2024): 100 harmful questions and answer targets, with topics covering risk categories in OpenAI's usage policy. The jailbreak judge's system prompt also comes from this benchmark.
   • GCG (Zou et al., 2023) is the only suffix algorithm run. The paper does not set other attack methods as comparison baselines.

   The authors state that, relative to the image-side theory and diverse jailbreak strategies above, this paper conducts a statistical and intervention analysis of language-side suffix transfer, to understand at a finer granularity when and why transfer succeeds.

   Collapse this answer

3. 03How does the paper address this problem?

   Define geometric features using the refusal direction, use mixed-effects logistic regression to relate transfer, then perform two types of interventions.

   Using the refusal direction to turn prompts and suffixes into regressable geometric quantities, then testing whether these quantities can improve GCG suffix transfer success through intervention.

   Transfer settings and success definition

   • Within-model transfer: whether suffix s, optimized for prompt p, succeeds on another prompt p′ of the same model.
   • Cross-model transfer: whether s, optimized for prompt p and model m, succeeds on another model m′; the prompt can remain p or be changed to p′. The prompt used for optimization is called the source prompt, and the one used for evaluation is called the target prompt.
   • ASR: ASR(s):=n_s^jailbroken/n_s^total, i.e., the number of prompts jailbroken after appending s divided by the number of prompts tested with s.
   • Plot filtering: In Figures 1, 2, and 6, the prompt axis only retains prompts that do not jailbreak without a suffix. The columns of Figure 2 are also limited to suffixes whose source prompt does not jailbreak without a suffix on the source model.

   Refusal direction and four features

   Feature activations and the refusal direction are taken at the same layer. The layer is selected as the optimal layer according to the effectiveness criterion of Arditi et al. (2024) (Appendix A, Definition 7). The paper does not give the layer number for each model. The refusal direction uses the residual stream of the last token at that layer; the refusal connection uses the end-of-instruction token.

   • Refusal direction (Definition 2): the difference between the average activations of harmful and harmless prompts at the last token of that layer.
   • Semantic similarity (Definition 3): the cosine similarity of the embeddings of two prompts. Embeddings come from two sources: the model's own activations, and the sentence embedding model all-mpnet-base-v2.
   • Refusal connection (Definition 4): the dot product of the activation of a harmful prompt without a suffix with the refusal direction, and their cosine.
   • Suffix push (Definition 5): Δ^push_ij:=⟨ a^base_i,v_refusal⟩−⟨ a^suffix_ij,v_refusal⟩, i.e., the decrease in the dot product of the activation with the refusal direction after appending the suffix.
   • Orthogonal offset (Definition 6): the L2 norm of the difference between activations before and after appending the suffix, after removing the projection onto the refusal direction.

   Section 3.2 states three hypotheses: prompts more aligned with the refusal direction transfer less easily; suffixes causing antiparallel offsets transfer more easily; prompts causing orthogonal offsets transfer more easily. The subject of the third hypothesis in the original text is the prompt, while Definition 6 defines the orthogonal offset on the suffix.

   Statistical analysis

   • Semantic similarity: each observation is (source prompt, suffix, target prompt), and the label is whether the target prompt is jailbroken by that suffix. Excluding same-prompt pairs, N=100×P(P−1)=990,000 per model. Cosine similarity is standardized to mean 0 and variance 1. A generalized linear mixed model with logit link is fitted using R's lme4. Random intercepts are the target prompt, and suffixes nested within source prompts.
   • Geometric quantities: observations are changed to prompt–suffix pairs, excluding the target prompt for which the suffix was originally optimized, with N still 990,000. Refusal connection, suffix push, and orthogonal offset are first regressed separately. The joint model then includes these three quantities and their pairwise products to estimate main effects and interactions; predictors are likewise standardized.
   • Cross-model samples: Table 3 states N=9,900 for each of the two directions. The paper does not explain how these 9,900 are drawn from the 10,000 suffixes per model. Appendix Table 8 additionally includes semantic similarity based on the model's internal embeddings in the joint model.

   Interventions and jailbreak determination

   • Prompt rewriting: Vicuna generates 10 meaning-preserving rewrites for each prompt, the dot product with the refusal direction is computed, and the relationship between the change in dot product and the change in ASR is examined. Appendix C gives the rewriting template; the template wording is not repeated here.
   • Modifying the GCG loss: a regularizer is added to the loss, either biasing toward pushing away from the refusal direction, or increasing the offset orthogonal to the refusal direction. Six nonzero coefficients are evaluated on Llama 3.2. There are 20 prompts total, randomly sampled 2 from each of the 10 JailbreakBench categories, and none jailbreak without a suffix. The suffix push term uses 100 suffixes per prompt, with 40,000 prompt/suffix pairs per coefficient; the orthogonal term uses 5 suffixes per prompt due to computational limits, with 2,000 pairs per coefficient. The paper does not give the regularizer formula.
   • Determination: Llama-3-Instruct-70B, with the system prompt taken from JailbreakBench. The choice is based on the recommendation of Chao et al. (2024, Table 1) for six common judges; the authors did not rerun those six judges in this paper.

   Collapse this answer

4. 04What experiments did the paper conduct?

   In Table 3, suffix push is mostly the largest main effect, but on Vicuna the refusal connection is larger.

   Experimental setup

   • Models tested: Qwen-2.5-3B-Instruct, Llama-3.2-1B-Instruct, Vicuna-13B-v1.5, Llama-2-7B-Chat. Table 6 states the alignment training as SFT/DPO/GRPO, SFT/DPO/RLHF, SFT, and SFT/RLHF, respectively, with sizes of 3B, 1B, 14B, and 7B. The model name in the main text is Vicuna-13B-v1.5, while Table 6's Model size is 14B. The authors state Vicuna is easier to jailbreak and Llama-2 is harder.
   • Data and suffixes: 100 harmful questions and answer targets from JailbreakBench. For each prompt, 100 suffixes are generated with GCG using different random seeds, giving 10,000 per model. The paper does not report the number of GCG steps, suffix length, or learning rate.
   • Metrics: ASR (Definition 1); standardized log odds from mixed-effects logistic regression, marginal R² (Rm², fixed effects only), and conditional R² (Rc², fixed plus random effects). Asterisks correspond to p<0.05, p<0.01, p<0.001.
   • Judge: Llama-3-Instruct-70B. Appendix B additionally performs human verification: 2,000 responses are drawn equally from each of the four models; by judge label, at most 2 jailbroken and 3 non-jailbroken per prompt per model, falling back to 5 non-jailbroken if the judge labels none as jailbroken, yielding 557 jailbroken and 1,443 non-jailbroken. One independent research assistant rechecked, with 98.65% agreement. The authors state this provides evidence against systematic labeling errors.
   • Regression samples: within-model, after excluding same-source pairs, 990,000 per model. Cross-model in Table 3 is 9,900. Semantic similarity uses the model's own activations and all-mpnet-base-v2; Table 1 labels the two rows as Model and Indep., and the table note does not state that Indep. is the latter.
   • Plot sampling: Figure 1 uniformly randomly draws 1 suffix from the 100 for each source prompt; quantitative analysis uses all suffixes. Figure 6 is a prompt × random seed matrix, with the same filtering rule as Figure 1.

   Main results

   Table 3 is a joint regression with interactions. The table below lists only main effects and Rm²; within-model N=990,000, and the last two cross-model rows have N=9,900. Unstarred coefficients are retained as in the paper's original table.

   Swipe horizontally to read the table

   Setting	Refusal connection	Suffix push	Orthogonal offset	Rm²
   Qwen	-1.29 (p<0.001)	3.05 (p<0.001)	0.29 (p<0.001)	0.688
   Vicuna	-4.85 (p<0.001)	3.45 (p<0.001)	-0.78 (p<0.001)	0.495
   Llama 2	-1.38 (p<0.001)	3.61 (p<0.001)	-0.28 (p<0.001)	0.592
   Llama 3.2	-0.18 (unstarred)	1.06 (p<0.001)	0.53 (p<0.001)	0.194
   Llama 3.2→Qwen	-1.26 (p<0.001)	1.34 (p<0.001)	0.53 (p<0.01)	0.354
   Qwen→Llama 3.2	-1.64 (unstarred)	-0.99 (unstarred)	1.59 (unstarred)	0.040

   Per Table 3. The corresponding Rc² values are 0.837, 0.871, 0.833, 0.661, 0.666, 0.983.

   • The authors state the main effects are broadly consistent with the univariate results: the higher the refusal connection, the lower the probability of transfer success; this negative effect is marked p<0.001 on Qwen, Vicuna, and Llama 2, while Llama 3.2's coefficient is unstarred. The higher the suffix push, the higher the success probability, marked p<0.001 within all four models; the authors state its effect is largest except on Vicuna, where refusal connection is most important.
   • Orthogonal offset is also marked significant, but its direction varies by model: positive on Qwen and Llama 3.2, negative on Vicuna and Llama 2. The authors state its role is smaller than the other features. Interactions are small relative to main effects, and the main text mainly interprets main effects.
   • Cross-model: the authors state the main effects of Llama 3.2→Qwen largely follow the pattern within Qwen. For Qwen→Llama 3.2, no statistically starred effects were found; the authors consider this very likely because the overall success rate in that direction is very low (Figure 2b), so the variance is insufficient to capture relationships.

   Semantic similarity and single features

   • Table 1: the standardized coefficients of cosine similarity are all positive and all marked p<0.001. Model row: Qwen 0.651, Vicuna 0.058, Llama 2 0.622, Llama 3.2 0.267. Indep. row: 0.226, 0.041, 0.192, 0.159. The authors write Rm²=0.000–0.048 and, following the rule of thumb of Chen et al. (2010), state most effect sizes are small. The authors state the large gap between Rm² and Rc² means most explained variance comes from the random effects of prompts and suffixes, not semantic similarity.
   • Table 2 single features, N=990,000: refusal connection is negative and marked p<0.001 only on Vicuna (-1.54); Qwen 0.31, Llama 2 0.08, and Llama 3.2 0.07 are unstarred. This differs from Qwen's negative coefficient in Table 3, and the paper does not explain the sign difference. Suffix push is 2.59, 2.87, 3.56, 1.28, and orthogonal offset is 1.44, 0.08, 2.54, 1.06; all eight coefficients are marked p<0.001.

   Intervention experiments

   • Rewriting: on Qwen, the correlation between the change in dot product and the change in ASR is -0.08 (p<0.05); on Llama 3.2 it is -0.18 (p<0.05). The authors expect a higher dot product to make transfer harder and ASR lower, and state that this relationship on the two models shows the statistical conclusions can guide interventions. Rewrites are generated by Vicuna; the paper does not report this correlation on Vicuna or Llama 2.
   • Suffix push regularization (Table 4, Llama 3.2, 40,000 pairs per coefficient): the ASR at coefficient 0 is 0.0138 (552 jailbreaks). Among the six nonzero coefficients, 0.001 is highest, with ASR 0.0214 (855). Coefficient 0.1 gives 0.0093 (373) and 0.5 gives 0.0101 (406), both lower than at coefficient 0.
   • Orthogonal offset regularization (Table 5, 2,000 pairs per coefficient): the ASR at coefficient 0 is 0.0145 (29). 0.00001 is highest, with ASR 0.0265 (53). Coefficient 0.1 gives 0.0020 (4) and 0.5 gives 0.0010 (2). The authors state that in both experiments the highest ASR occurs at a nonzero coefficient, consistent with the statistical analysis. The denominators of the two tables differ, so the ASRs should not be compared directly across tables.

   Qualitative transfer

   • The authors state that in Figure 1, Qwen's within-model success pattern is the densest, Llama 3.2 is noticeably sparser, Vicuna and Llama 2 are in between, and success is concentrated in some prompts and suffixes rather than uniformly distributed across the matrix. Figure 6 is used to show that this model difference does not come only from the single suffix drawn in Figure 1. The main text does not give the proportion of colored cells.
   • The authors state that some prompts form denser horizontal bands and some suffixes form denser vertical bands. There is also an off-target phenomenon: a suffix optimized for one prompt may fail to break that prompt but can break other prompts of the same model.
   • The authors state Figure 2 is asymmetric: suffixes optimized on the more aligned Llama 3.2, as the authors call it, transfer to the less aligned Qwen better than in the reverse direction. In Figure 4, the authors state that the average harmful prompt activation (blue line) has the highest cosine with the refusal direction; the three least successful suffixes (orange lines) only slightly reduce that cosine, while the three most successful ones (green lines) push the similarity down. In Figure 5 and Appendix Figures 7–9, the authors state that the higher the suffix push and orthogonal offset, the higher the ASR computed over all tested prompts; the signal is weaker on the Llama models, which the authors attribute to fewer successful transfer examples. In Figure 3, the authors state that Vicuna's cosine and normalized dot product distributions are the most spread out, and consider this a possible explanation for why it answers some harmful questions even without a suffix.

   Other ablations and analyses

   • Table 7, Qwen interactions: refusal connection × suffix push is 0.39 (p<0.001), refusal connection × orthogonal offset 0.69 (p<0.001), suffix push × orthogonal offset -0.91 (p<0.001); intercept -3.51 (p<0.001).
   • Table 7, Vicuna interactions: 0.51, -0.25, -0.06, all p<0.001; intercept -4.93 (p<0.001). Llama 2 interactions: 0.09 (p<0.01), -0.29 (p<0.001), 0.40 (p<0.001); intercept -7.46 (p<0.001).
   • Table 7, Llama 3.2 interactions: -0.19 (p<0.001), 0.01 (unstarred), 0.02 (p<0.05); intercept -6.56 (p<0.001). The three interactions for Llama 3.2→Qwen, 0.03, 0.19, -0.07, are all unstarred, with intercept -6.47 (p<0.001).
   • Table 7, Qwen→Llama 3.2 interactions -0.86, 1.36, 0.52 are unstarred, with intercept -23.25 (p<0.001). The authors caution against overinterpreting signs when interactions are much smaller than main effects.
   • Table 8, after adding semantic similarity based on the model's internal embeddings, its coefficients are Qwen 0.25, Vicuna -0.05, Llama 2 0.62, Llama 3.2 0.25, all p<0.001. The authors state it is positive except on Vicuna, and its influence is small relative to suffix push and refusal connection. The Rm² values in that table are 0.693, 0.493, 0.608, 0.205.
   • In Table 4, the ASRs for coefficients 0.00001, 0.0001, 0.01 are 0.0177, 0.0189, 0.0176; in Table 5, those for 0.0001, 0.001, 0.01 are 0.0195, 0.0175, 0.0115.

   Collapse this answer

5. 05What are some points that can be further explored?

   The paper does not specifically discuss limitations; the experiments state four open-source models, GCG, and JailbreakBench.

   Limitations and future directions noted by the authors

   The paper does not specifically discuss limitations. The Conclusion states the authors hope these insights can be used to develop stronger defenses, but it is not framed as pending experiments, nor are shortcomings listed separately.

   Experimental coverage

   • Tested models are Qwen-2.5-3B-Instruct, Llama-3.2-1B-Instruct, Vicuna-13B-v1.5, Llama-2-7B-Chat, 4 in total; each generates 10,000 suffixes with GCG (Section 4, Table 6).
   • Prompts come from 100 harmful questions from JailbreakBench. Within-model GLMM, after excluding same-source pairs, yields 990,000 observations per model (Tables 1–3).
   • Cross-model regression only reports Llama 3.2→Qwen and Qwen→Llama 3.2, each N=9,900 (Table 3). The paper does not explain how these 9,900 are sampled relative to the 10,000 suffixes per model.
   • Jailbreak determination uses Llama-3-Instruct-70B; Appendix B manually checks 2,000 examples, with 98.65% agreement. Refusal direction takes the optimal layer; the paper does not report the layer numbers selected for each model, nor the number of GCG iterations or suffix lengths.
   • Correlation coefficients for rewriting experiments are only given for Qwen and Llama 3.2. Loss modification runs only on 20 prompts from Llama 3.2: suffix push has 40,000 pairs per coefficient, orthogonal offset has 2,000 pairs per coefficient (Section 5.6, Tables 4–5). Embedding comparisons use the model's own activations and all-mpnet-base-v2 (Section 3.3, Table 1).

   Collapse this answer

6. 06Summarize the main content of the paper

   Across four models, the relationship between GCG suffix transfer and the magnitude of pushing away from the refusal direction is most stable; semantic similarity is only weakly correlated.

   This work uses statistics and activation geometry to analyze when GCG adversarial suffixes transfer to new prompts or models.

   • Problem: short, meaningless suffixes appended to harmful requests often elicit prohibited content on prompts and models not involved in optimization. The authors believe transfer is empirically established, but a fine-grained explanation of the conditions is lacking.
   • Approach: on the 100 harmful questions in JailbreakBench, they generate 10,000 GCG suffixes each for Qwen-2.5-3B-Instruct, Llama-3.2-1B-Instruct, Vicuna-13B-v1.5, and Llama-2-7B-Chat. Along the refusal direction of Arditi et al. (2024), they define refusal connection, suffix push, and orthogonal offset, and compare them with semantic similarity from two embeddings. Within-model transfer uses logistic mixed-effects models with 990,000 observations; jailbreaks are judged by Llama-3-Instruct-70B, with 98.65% agreement with humans on 2,000 samples.
   • Regression: in Table 3, suffix push is marked p<0.001 within all four models, with a coefficient of 3.61 for Llama 2; the authors state it is the largest main effect except on Vicuna. Vicuna's refusal connection is -4.85, and the authors state this prompt-side factor is most important on that model. The sign of orthogonal offset varies by model. Semantic similarity has marginal R² of 0.000–0.048 (Table 1), and the authors state the effect size is small.
   • Cross-model and interventions: the main effects of Llama 3.2→Qwen are similar to those within Qwen; Qwen→Llama 3.2 has no starred main effects, with Rm² of 0.040, which the authors consider very likely due to the success rate being too low. After rewriting prompts to be closer to or farther from the refusal direction, the correlation between the change in dot product and the change in ASR is -0.08 and -0.18 on Qwen and Llama 3.2, respectively (both p<0.05). After modifying the GCG loss on Llama 3.2, the suffix push coefficient 0.001 gives an ASR of 0.0214, higher than the 0.0138 at coefficient 0; the orthogonal term coefficient 0.00001 gives an ASR of 0.0265, higher than 0.0145. When the coefficient is too large, the ASR is lower than at coefficient 0.
   • Authors' conclusions: suffix push is the most important suffix-side factor across models, refusal connection is an important prompt-side factor on some models, and semantic similarity is only weakly correlated. The authors state these quantities can be used to design stronger attacks, and they hope they can also be used for stronger defenses.

   Collapse this answer

Related papers