Skip to content

Endogenous preferences as vulnerability signals: discovering universal adversarial trigger words that degrade LLM utility

Endogenous Preferences as Vulnerability Signals: Discovering Universal Adversarial Triggers for LLM Utility Degradation

arXiv: 其他 AI 安全论文· arXiv:2611.18592· Qingyu Wu, Zeyu Feng, Yuzhe Luo, Hua ChengThe Defense Innovation Institute, Academy of Military SciencesIndexed by this site · 2 days ago· Originally publishedAttacks· Its place in the scan record →

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors propose EndoUT: without specifying a downstream failure form and without task feedback, it selects fixed trigger segments solely from the clean behavior of a frozen victim model; across 4 models and 9 benchmarks it reduces utility by an average of 40.2 percentage points.

Threat modelThe attacker can place a bounded segment in the user-visible prompt context (token budget L, default 32 generator tokens); the victim model is frozen, and benign requests and downstream task protocols are unchanged. The attacker can query the victim model's next-token probabilities on unlabeled source instructions; the discovery stage has no externally specified target, benchmark input or label, task scorer, judge feedback, or utility metric. The main protocol appends the trigger word as a user-content suffix, and prefix and random-position variants test sensitivity to insertion position.

Questions
Adversarial prompting and red-teaming methods usually rely on task labels, harmful goals, judge feedback, or downstream utility scores to define the failure they seek. The authors ask: without knowing the task-level form of the failure and without downstream feedback, can the clean behavior of a frozen model itself expose reusable context vulnerabilities.
Methods
EndoUT uses a Siamese-style two-pass comparison of the same frozen victim model on clean input and after inserting a segment: taking the preferred response-onset token of the clean pass as reference, it uses EPD to measure the suppression of that token's probability by the insertion; an auxiliary generator (trained with GRPO) proposes bounded segments, EPD ranking selects one fixed trigger per victim model, which is then reused verbatim on all held-out inputs.
Experiments and results
Across 4 instruction-tuned models and 9 benchmarks, fixed triggers reduce utility by an average of 40.2 percentage points, with 35 of 36 cells negative, and the average drop exceeds all evaluated baselines; transfer appears in all 24 victim–generator pairs but with varying strength; regressions in 19/24 pairs concentrate in a single output cluster, and the dominant failure category varies by victim model.
Limitations and future work
The authors note: the experiments cover only the realized proposer pools rather than the entire bounded segment space, and generated strings become concentrated during training; the analysis reveals recurring failure manifestations but does not perform causal interventions; the implementation requires next-token probabilities of open-source models, and extension to restricted-access models remains unresolved. The evaluated models are 4 open-source instruction-tuned models.
Experimental setupQwen2.5-7B-Instruct、Llama-3.1-8B-Instruct et al. · Databricks Dolly 15K、MMLU et al. · ∆Utility (attacked-minus-clean, percentage points)、EPD (Endogenous Preference Disruption) et al.
Evaluated models
Qwen2.5-7B-InstructLlama-3.1-8B-InstructMistral-7B-Instruct-v0.3Gemma-2-9B-IT
Benchmarks
Databricks Dolly 15KMMLUMMLU-ProGSM8KMATH-500BBHHellaSwagARCTruthfulQAIFEval
Metrics
∆Utility (attacked-minus-clean, percentage points)EPD (Endogenous Preference Disruption)Retention@K
AI summaryThe authors propose the EndoUT method, which, without task labels or downstream feedback, mines universal adversarial trigger words for LLMs that can be reused across inputs, reducing benign task utility by an average of 40.2 percentage points across four instruction-tuned models and nine benchmarks. The method uses a twin-style design to compare the behavior of the same frozen model on the original instruction versus a version with the inserted segment, measures with endogenous preference disruption (EPD) the degree to which the insertion weakens the model's preference for the starting token of its preferred response, and has an auxiliary generator propose candidate segments, ranked by score. The highest-scoring segment is fixed before evaluation and reused verbatim across all held-out inputs; the average utility change is negative on every benchmark, and under the reported protocol the average decline is larger than all evaluated baselines. Ablations show that selecting on the source side first and then fixing for reuse transfers better than fixed reuse with an empty context or per-input generation, and the association between trigger-word families and downstream failure modes varies by victim model. Code and data will be released upon paper acceptance.

The authors propose the EndoUT method, which, without task labels or downstream feedback, mines universal adversarial trigger words for LLMs that can be reused across inputs, reducing benign task utility by an average of 40.2 percentage points across four instruction-tuned models and nine benchmarks. The method uses a twin-style design to compare the behavior of the same frozen model on the original instruction versus a version with the inserted segment, measures with endogenous preference disruption (EPD) the degree to which the insertion weakens the model's preference for the starting token of its preferred response, and has an auxiliary generator propose candidate segments, ranked by score. The highest-scoring segment is fixed before evaluation and reused verbatim across all held-out inputs; the average utility change is negative on every benchmark, and under the reported protocol the average decline is larger than all evaluated baselines. Ablations show that selecting on the source side first and then fixing for reuse transfers better than fixed reuse with an empty context or per-input generation, and the association between trigger-word families and downstream failure modes varies by victim model. Code and data will be released upon paper acceptance.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Without specifying a downstream failure form and without task feedback, can the clean behavior of a frozen model expose a fixed trigger segment that transfers across tasks.

    Core question: without specifying a downstream failure form and without downstream task feedback, can the clean behavior of a frozen victim model reveal a reusable contextual vulnerability (a fixed inserted segment) that transfers across inputs and tasks.

    • Problem setting and importance: deployed LLMs must process untrusted text in the prompt context; an inserted segment need not produce prohibited content, but may disrupt reasoning, answer format, or instruction following, reducing utility on benign tasks. If the same segment causes loss across unrelated requests, it exposes a reusable contextual vulnerability.
    • Shortcomings of existing methods (per the paper): adversarial search and robustness evaluation usually first specify the behavior to induce or the task-level criterion to reduce—universal-trigger methods define objectives with labels, target classes or tokens, and task losses, and automated jailbreaks reward harmful compliance, refusal bypass, or judge success; these approaches write "what counts as task-level failure" into the search objective in advance.
    • Core observation: the victim model's preferred token at response onset satisfies three requirements—it is available for every unlabeled instruction, it is conditioned on that instruction, and it does not specify any downstream error; therefore candidates can be compared on the same instruction by how much the inserted segment suppresses that token's probability.
    • The paper proposes: EndoUT, a Siamese-style two-pass design that uses Endogenous Preference Disruption (EPD) to score and rank candidate segments on source-side unlabeled instructions, selects one fixed trigger word per victim model, and then reuses it verbatim on held-out benchmarks; the authors report an average utility drop of 40.2 percentage points across 4 models and 9 benchmarks.
    • Threat model:
      • Goal: find a fixed segment that transferably reduces benign utility on held-out inputs and tasks; the discovery stage does not specify a failure form.
      • Knowledge: the attacker can query the frozen victim model's next-token probabilities on unlabeled source instructions; there is no externally specified target, benchmark input or label, task scorer, judge feedback, or utility metric.
      • Capability: place a bounded segment in the user-visible prompt context (token budget L, default 32 generator tokens); the main protocol appends it as a user-content suffix, with prefix and random-position as variants.
      • Victim system: a frozen open-source instruction-tuned LLM, with benign requests and downstream task protocols unchanged.
  2. What related research exists?

    The paper positions its work at the intersection of three lines of research: textual adversarial attacks, utility/usability failures, and prompt robustness; the difference is that the discovery stage uses no task-level signal.

    Textual adversarial attacks

    • Wallace et al. (2019), Jin et al. (2020), Li et al. (2020), Boucher et al. (2022), etc.—optimize discrete perturbations under task-specified objectives, from character/word-level edits against supervised models to input-agnostic token sequences.
    • Parekh et al. (2021), Xu et al. (2022), Xu and Wang (2024), Meade et al. (2024), Shin et al. (2020), Wen et al. (2023)—reduce reliance on labeled samples, study universal/natural triggers in prompt-based models, or search hard prompts using target tokens, answer spans, labels, or task loss. The paper notes that although these untargeted objectives need not specify a concrete erroneous output, EndoUT further gives no downstream task feedback and measures utility only after fixing victim-specific segments.
    • Zou et al. (2023), Liu et al. (2023), Chao et al. (2023), Mehrotra et al. (2023), Yu et al. (2023), Liao and Sun (2024), Paulus et al. (2024), Sadasivan et al. (2024)—automated attacks against aligned LLMs, searching adversarial suffixes, semantic prompts, iterative pipelines, fuzzing strategies, or learned generators, with the goal of harmful compliance or refusal bypass; JailbreakBench, HarmBench, StrongREJECT (Mazeika et al. 2024) standardize such evaluations with harmful behavior datasets and judges.

    LLM utility and usability failures

    • Greshake et al. (2023), Li, Guo, and Cai (2025)—prompt injection exploits instruction hierarchy, untrusted context, or the system-prompt channel to redirect model behavior, and can cause utility loss without involving harmful content.
    • Zhang, Xiong, and Mao (2024), Gao et al. (2024)—guardrail false positives and poisoning-style denial of service; Manu et al. (2025), Liu et al. (2026)—over-generation and pathological reasoning cost; Zhang et al. (2026)—soft failures of RAG systems.
    • Mai et al. (2025)—a safety–utility trade-off benchmark, showing that guardrail mechanisms themselves reduce task performance.

    Prompt robustness

    • Zhu et al. (2024)—PromptBench, studies predefined character-, word-, sentence-, and semantic-level prompt perturbations.
    • Cao et al. (2024), Seleznyov et al. (2025), Zheng et al. (2025)—worst-case prompts, sensitivity to punctuation, formatting, and prompt components, documenting heterogeneous sensitivity across task forms.

    Baselines and benchmarks: the compared baselines are PromptBench, UAT, GCG, BEAST, AmpleGCG, AdvPrompter, System Prompt Poisoning, and a hand-written pool (verbosity, formatting, caution, distraction instructions); the evaluation benchmarks are MMLU, MMLU-Pro, GSM8K, MATH-500, BBH, HellaSwag, ARC, TruthfulQA, IFEval, 9 in total.

    The authors position this paper as follows: all the above methods first specify a failure form or a feedback channel, whereas EndoUT specifies neither a downstream failure nor uses downstream feedback during discovery, so the held-out utility change measures transfer rather than direct optimization.

  3. How does the paper address this problem?

    EndoUT uses the clean pass's preferred onset token as reference, scores and ranks candidate segments on the source side with EPD, selects one fixed trigger word, and then reuses it verbatim on held-out tasks.

    Overall approach: separate "discovery" from "evaluation"—the source side uses only the frozen victim model's clean behavior to score candidate segments, and only after selecting one victim-specific fixed trigger word is utility change measured on held-out benchmarks; since neither discovery nor selection sees benchmark feedback, the drop on held-out is interpreted as transfer rather than direct optimization. The method is named EndoUT, and the core score is Endogenous Preference Disruption (EPD).

    Problem setting and formalization

    • The victim model fθ is frozen; for a benign instruction x and candidate segment z (token budget L), the main protocol's insertion operator appends z to the end of the user content, Tz (x) = x⊕z.
    • Benchmark utility is defined as the mean of official normalized scores converted to percentage points: Ub(fθ) = 100 · 𝔼ξ∼ Db[sb(fθ(cb(ξ)), ξ)]; utility change is ∆Utility = attacked − clean, with negative values indicating degradation.
    • The discovery stage has no externally specified target token or response, held-out benchmark input or label, task scorer, judge feedback, or utility metric; task-level information enters only after z is fixed.

    Clean behavior as discovery reference (EPD)

    • For an unlabeled source instruction u, take the clean pass's preferred token at response onset ŷ(u) = arg max pθ(y | cθ(u)) as the reference conditioned on that instruction; it does not specify any alternative answer.
    • EPD measures the suppression of this reference by the insertion: rEPD(z; u) = ℓz(u) − ℓ0(u), where ℓ0 and ℓz are the negative log probabilities of the reference token under clean and inserted conditions respectively; EPD is positive when the insertion lowers the preferred token's probability.
    • The two passes use the same frozen fθ and the same chat template, differing only in the inserted segment, forming a within-instruction paired comparison; the paper states that this two-pass construction is analogous to shared-parameter comparison in Siamese/contrastive learning, but compares response behavior rather than learned representations.

    Proposal search and fixed trigger word selection

    • Each victim model is paired with an auxiliary generator gϕ, which samples candidate segments for each source instruction under a fixed meta-instruction (specifying phrase form and token budget L); GRPO pushes the proposal distribution toward segments with higher source-side scores, updating only gϕ while fθ stays frozen.
    • The source corpus is the instruction field of Databricks Dolly 15K, 14,824 unlabeled instructions after deduplication; training stops after the EPD plateau is smoothed, the checkpoint with the highest training EPD is taken, and each source instruction yields one candidate, forming an index pool Cv (duplicate strings produced by different instructions are retained).
    • Candidates are ranked by EPD only on their source instruction, not by cross-input effectiveness or downstream utility; the (u, z) with the highest EPD across the pool is taken as that victim model's fixed trigger word zv⋆, after which gϕ is no longer used and zv⋆ is inserted verbatim into all held-out inputs for that victim model.
    • The default generator is Qwen3-1.7B, with a trigger word budget of 32 generator tokens; sampling temperature 1.0, top-p 0.9, repetition penalty 1.2.

    Evaluation method

    • 4 open-source instruction-tuned victim models: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT.
    • 9 benchmarks each use their official score as utility, reporting attacked-minus-clean percentage-point change; on the public grid, each column averages over the 4 victim models, and the overall average covers all 36 victim–benchmark cells.
    • Each baseline is evaluated on the same grid with its native objective, prompt channel, and final artifact: UAT gives a fixed prefix, GCG/BEAST/AmpleGCG give a fixed suffix, AdvPrompter gives an input-conditioned suffix; the PromptBench pool and handwritten pool receive benchmark-specific oracle selection in each victim–benchmark cell; System Prompt Poisoning serves as a system-channel availability reference.
  4. What experiments did the paper conduct?

    The fixed trigger word reduces utility by an average of 40.2 points across 4 models × 9 benchmarks, with 35 of 36 cells negative; transfer reproduces across proposal models but with varying strength, and regressions are highly concentrated within victim–trigger pairs.

    Experimental Setup

    • Victim models: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, 4 open-source instruction-tuned models in total.
    • Benchmarks: MMLU, MMLU-Pro, GSM8K, MATH-500, BBH, HellaSwag, ARC, TruthfulQA, IFEval, 9 in total, covering knowledge, mathematical reasoning, commonsense reasoning, truthfulness and instruction following; the source corpus is 14,824 unlabeled instructions after deduplicating Dolly 15K.
    • Baselines: PromptBench, UAT, GCG, BEAST, AmpleGCG, AdvPrompter, System Prompt Poisoning, hand-written pool; among them, the PromptBench pool and the hand-written pool obtain benchmark-specific oracle selection in each victim–benchmark cell.
    • Metric: ∆Utility = post-attack − clean utility (percentage points, negative values indicate degradation); in parentheses is the average relative drop per victim model, 100(Uclean − Uattack)/Uclean.
    • Default configuration: auxiliary generator Qwen3-1.7B, trigger budget 32 generator tokens, sampling temperature 1.0, top-p 0.9, repetition penalty 1.2; GRPO updates only the generator.
    • Review method: failure category analysis was independently reviewed by 3 annotators on 1,000 benchmark-stratified regressions (250 per victim model), using a six-category codebook, labels required full agreement, disagreements were independently relabeled; utility scoring used each benchmark's official scorer, and the paper does not report using an LLM judge for utility evaluation.

    Main Results

    According to Table 1 (values are ∆Utility averaged over the 4 victim models, in percentage points; in parentheses is the average relative drop):

    Swipe horizontally to read the table

    MethodMMLUGSM8KBBHARCTruthfulQAIFEvalAvg.
    PromptBench-2.3-7.6-14.2-8.3-4.2-5.9-6.5
    GCG-26.1-33.8-37.9-37.9-12.8-16.7-24.3
    AdvPrompter-1.1-6.4-7.6-1.7-0.7-5.7-3.4
    System Prompt Poisoning-9.3-15.9-6.0-55.2-36.80.6-19.1
    Hand-written-1.4-34.3-31.5-1.1-2.4-20.3-12.1
    EndoUT (suffix)-36.2-58.4-43.6-62.4-33.2-33.1-40.2
    EndoUT (prefix)-19.1-24.4-24.4-45.8-28.1-26.2-25.1
    EndoUT (random)-21.9-30.8-31.2-45.3-25.1-29.8-27.3

    (The table omits the three rows UAT, BEAST, AmpleGCG, whose Avg. are 0.4, 0.3, -1.8 respectively; it omits the three columns MMLU-Pro, MATH-500, HellaSwag.)

    • EndoUT suffix triggers average -40.2 points, 35 of the 36 victim–benchmark cells are negative, and the 4-victim-model mean is negative on every benchmark; the authors state that since neither discovery nor selection sees benchmark feedback, this pattern indicates that the segments selected on the source side produce degradation that transfers across inputs and across tasks, rather than direct optimization against the evaluated results.
    • The authors state that GCG is the strongest fixed user-suffix baseline (-24.3); System Prompt Poisoning averages -19.1 and exceeds EndoUT only on TruthfulQA; even with benchmark-specific oracle selection, PromptBench and the hand-written pool average only -6.5 and -12.1.
    • Position variants: prefix and random-position insertion average -25.1 and -27.3 respectively (discovery, selection and evaluation are redone for each position); the authors accordingly state that transfer is not limited to suffix insertion, but the magnitude varies with the position protocol, and the suffix setting is the largest.

    Protocol and Selection Ablations (Table 2)

    • Effect of source-side selection: null-context fixed prefix (generated from one fixed placeholder and reused) is -16.26, per-input prefix generation is -14.33, while the matched source-side selection fixed-prefix protocol is -25.1 (Table 1); the authors state that this matched comparison attributes the stronger result to selection over unlabeled source instructions, rather than to fixed reuse or input-specific conditioning alone.
    • Length control: the 16-token four-victim-model run is -20.89, the 64-token Qwen-only run is -26.12, compared with their respective 32-token references of -40.20 and -66.5; the authors state that degradation is non-monotonic with budget, arguing against a "just length" explanation.

    Transfer Breadth of Source-Side Ranking (Table 3)

    • The source-side Top-20 deduplicated candidates for the 4 victim models (80 in total) all have negative average held-out ∆U; at K=20, the average EPD retains 82.8–99.0% of Top-1's, and the average utility loss retains 84.5–111.0%.
    • For Mistral and Gemma, the average utility loss of Top-5 exceeds that of Top-1 (Mistral 101.5%, Gemma 114.9%), even though downstream utility never enters source-side ranking; the authors state that source-side ranking identifies a set of transferable candidates rather than an isolated optimum.

    Transfer Across Proposer Models (Table 4, Figure 2)

    • Redoing the full protocol with Qwen3-1.7B and 5 alternative proposer backbones (Qwen3-4B, Llama-3.2-1B, Llama-3.2-3B, Dolphin3.0-3B, victim-as-generator): the average ∆Utility is negative for all 24 victim–generator pairs, ranging from -1.1 for Llama-3.2-1B→Gemma-2-9B to -66.5 for Qwen3-1.7B→Qwen2.5-7B, a span of 65.4 points.
    • The authors state that the direction of transfer does not depend on the default proposer, but reproduction does not mean consistent recovery strength; the strongest proposer varies with the victim model, and neither generator scale (within Qwen3, Llama-3.2) nor using the victim model itself as generator yields a stable ordering.
    • Figure 2 uses Sentence-T5 space to measure the closeness of each proposer pool to post-hoc selected high-degradation anchors (E1–E3, with their utility loss annotated below each anchor, e.g., E1 for Qwen is 66.8 points): the frequency-weighted 95th-percentile cosine from each pool to the anchors is between 0.64 and 0.97; the authors state that this supports a limited-search explanation—recovery strength is a joint property of the context vulnerability and the candidate neighborhood exposed by the proposer.

    Structure and Semantics of Regressions (Figure 3, Figure 4)

    • For the 24 evaluated fixed suffixes, each victim–trigger pair has 150–2,237 paired regressions; in 19/24 pairs at least 70% of regressions fall within one output cluster, and 13 pairs exceed 90%; whereas after mixing all 6 triggers of each victim model, the largest output cluster accounts for only 19.8–48.5%. The authors state that recurrence is concentrated within victim–trigger pairs rather than inherited from the victim-level output distribution.
    • Among the 1,000 reviewed regressions (250 per victim model), the dominant category varies by victim model: Qwen 88.0% short collapse, Llama 61.2% wrong but well-formed answers, Mistral 96.8% off-task over-generation, Gemma split between format/parse failure (62.8%) and short collapse (36.4%). The authors state that failure forms are structured at the victim–trigger level and are not prescribed by the discovery objective.

    Other Ablations and Analyses

    • Proposer support contraction: on 1,000 fixed Dolly prompts, normalized Shannon effective support drops from 0.920–1.000 at the first checkpoint to 0.007–0.063 at the last, and Top-10 normalized string mass rises from 0.010–0.040 to 0.559–0.926 (main text).
    • The paper does not report statistical significance tests, multiple repeated runs, or variance.
  5. What are some points that can be further explored?

    The authors list three directions: proposal coverage, causal mechanism, and restricted-access models; experiments cover 4 open-source victim models, 9 benchmarks, and 6 proposal backbones.

    Limitations and Future Directions Noted by the Authors

    • Limited proposer coverage (Limitations): the experiments establish breadth within the realized proposer pools, not coverage of the entire bounded segment space; generated strings become increasingly concentrated during training, and alternative proposers recover widely differing held-out degradation under the same victim model and EPD criterion. The authors suggest that negative search results may reflect insufficient exploration rather than the absence of transferable segments.
    • Future direction one (Limitations): develop diversity-preserving search and coverage diagnostics to quantify how the set of recovered transferable segments expands as proposer support grows.
    • Future direction two (Limitations): the clustering and annotation analysis reveals recurring, pair-specific failure manifestations but does not establish the causal computation between response-onset perturbation and downstream generation; causal interventions on internal activations across triggers could test whether these regressions share the same pathway.
    • Future direction three (Limitations): the current implementation requires next-token probabilities of open-source victim models; extending endogenous discovery to restricted-access models remains unresolved.

    Scope of Experimental Coverage

    • The evaluated victim models are 4 open-source instruction-tuned models: Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT (Experimental Setup).
    • The evaluation benchmarks are 9 public benchmarks, and the source corpus is 14,824 unlabeled instructions after deduplicating Dolly 15K; the main results grid is 4 victim models × 9 benchmarks, 36 cells in total (Experimental Setup, Table 1).
    • Proposer backbones cover 6: Qwen3-1.7B, Qwen3-4B, Llama-3.2-1B, Llama-3.2-3B, Dolphin3.0-3B, victim-as-generator, forming 24 victim–generator pairs (Table 4).
    • Insertion positions cover three protocols: suffix, prefix, random-position; token budgets cover three levels: 16, 32, 64, among which the 64-token behavior is a diagnostic on the Qwen victim model only (Table 1, Table 2).
    • The paper does not report multiple repeated runs, confidence intervals, or statistical significance tests; the failure category review is 250 per victim model, 1,000 in total, requiring full agreement among 3 annotators (main text, Structure and Semantic Forms of Regressions section).
  6. Summarize the main content of the paper

    EndoUT selects fixed triggers solely from the frozen model's clean onset preference, reducing utility by an average of 40.2 points across 4 models × 9 benchmarks; transfer reproduces across proposers, and failure forms vary by victim model.

    An attack paper studying whether the clean behavior of a frozen LLM can expose reusable contextual vulnerabilities without specifying a downstream failure form and without any task-level feedback.

    • Problem: existing adversarial search writes "what counts as failure" into the objective in advance (labels, harmful targets, judges, task losses); the authors ask whether a fixed inserted segment that transfers across inputs and tasks can be discovered from clean behavior alone.
    • Method: EndoUT uses a Siamese-style two-pass comparison of the response-onset distribution of the same frozen victim model under clean and inserted-segment conditions, defining an EPD score with the clean pass's preferred token as reference; an auxiliary generator trained with GRPO (default Qwen3-1.7B, 32-token budget) proposes candidates over 14,824 unlabeled Dolly instructions, one fixed trigger word is selected per victim model by EPD, and then reused verbatim on held-out benchmarks.
    • Main results: across 4 victim models × 9 benchmarks, the fixed suffix trigger word has a mean ∆Utility of -40.2 percentage points, with 35 of 36 cells negative; the strongest fixed-suffix baseline GCG is -24.3, and the oracle-selected PromptBench pool and handwritten pool are only -6.5 and -12.1 (Table 1).
    • Transfer breadth: the source-side Top-20 candidates (per victim model, 80 total) all have negative mean ∆U; all 24 victim–generator pairs have negative mean ∆Utility, but strength varies from -1.1 to -66.5 (Table 3, Table 4).
    • Failure structure: in 19/24 victim–trigger pairs, at least 70% of regressions concentrate in a single output cluster; among 1,000 reviewed regressions, the dominant category varies by victim model (Qwen 88.0% short collapse, Mistral 96.8% off-task over-generation), and the authors state that failure forms are structured at the victim–trigger level and are not specified by the discovery objective.
    • Authors' conclusion: clean model behavior can serve as an endogenous discovery signal, and downstream failures should be treated as manifestations to be characterized rather than as pre-specified objectives; the authors also note that experiments cover only the realized proposal pool, do not perform causal interventions, and require next-token probabilities from open-source models, with extension to restricted-access models still unresolved. The authors plan to release code and data upon paper acceptance.

Related papers

Read originalarxiv.org