Skip to content

FlipJev: fast black-box red teaming finds universal adversarial suffixes for typed decision models

FlipJev: Fast Black-Box Red Teaming Finds Universal Adversarial Suffixes for Typed Decision Models

arXiv· arXiv:2610.38895· Xiaogeng Liu, Xinyan Wang, Chaowei XiaoJohns Hopkins University; University of Wisconsin–MadisonIndexed by this site · 21 days ago· Originally publishedAttacks

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors perform a black-box query attack on the typed decision model Jev: a universal suffix flips 43% of unseen documents on 9 datasets and transfers across tasks.

Threat modelThe attacker's goal is to flip Jev's correct decisions (e.g., yes to no, unsafe to safe). Black-box: can only read the probabilities Jev returns; can only append a suffix at the end of untrusted content, cannot change the developer's trusted question; the suffix contains no topic words of the question and is at most 32 o200k_base BPE tokens (64 for trajectory review).

Questions
Typed decision models (such as Jev) make decisions with a single cheap query and are used in safety scenarios, but existing adaptive attacks have not exploited their speed, and almost no one has studied universal adversarial examples that transfer across inputs and tasks.
Methods
The authors propose FlipJev: no attacker LLM; evolutionary search with word-level edits generates a short suffix, with fitness based only on Jev's returned probabilities; scoring by the average P(no) over 10 training documents yields a universal suffix.
Experiments and results
Common prompt injection had zero flips in 10,240 paired tests; per-document search flipped 58/180 targets across 9 datasets, the universal suffix flipped 43% of held-out documents, and the arXiv full-text suffix flipped 90–100% of documents across 5 datasets across tasks.
Limitations and future work
The authors state that attacks that change the semantics of the content belong to another threat model and are out of scope; the search component could be further streamlined to reduce queries, left as future work. Experiments were conducted only on the single model Jev and on English public datasets.
Experimental setupJev · email-comprehension dataset、TREC-QC et al. · ASR (flip rate)、mean safety score drop et al.
Evaluated models
Jev
Benchmarks
email-comprehension datasetTREC-QCAG NewsDBpedia-14Yahoo! AnswersPatent abstractsarXiv abstracts20 NewsgroupsLexGLUE ECtHRarXiv full textR-JudgeATBench500IMDB
Metrics
ASR (flip rate)mean safety score dropP(no)
AI summaryThe researchers propose FlipJev, a fast black-box query attack against typed decision models (language models that directly output a decision rather than free text). It does not call an attacker language model, and on the representative model Jev it can find short suffixes that flip high-confidence answers within minutes. Experiments cover 12 datasets across different data types and application scenarios. Results show that Jev resists common prompt injection: in 10,240 paired tests, no strategy changed the correct answer. However, the suffixes found by the attack are universal: one suffix per dataset flips 43% of unseen documents, and a suffix generated from one task flips 90-100% of samples on other tasks. The authors therefore argue that evaluation of typed decision models must include rigorous adaptive attacks.

The researchers propose FlipJev, a fast black-box query attack against typed decision models (language models that directly output a decision rather than free text). It does not call an attacker language model, and on the representative model Jev it can find short suffixes that flip high-confidence answers within minutes. Experiments cover 12 datasets across different data types and application scenarios. Results show that Jev resists common prompt injection: in 10,240 paired tests, no strategy changed the correct answer. However, the suffixes found by the attack are universal: one suffix per dataset flips 43% of unseen documents, and a suffix generated from one task flips 90-100% of samples on other tasks. The authors therefore argue that evaluation of typed decision models must include rigorous adaptive attacks.

Related events and news

1 events · 1 news mentions

News that mentions this paper

周报:预训练投毒可植入智能体后门

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    How to red-team the typed decision model Jev: exploit its speed for adaptive attacks, and search for universal adversarial suffixes that span inputs and tasks.

    Core problem: the adversarial robustness of typed decision models (represented by Jev) lacks rigorous evaluation—existing adaptive attacks do not exploit its millisecond response speed, and almost no one has searched for universal adversarial examples that can flip multiple inputs or even multiple tasks.

    • Setting and importance: typed decision models do not generate text but return per-option probabilities in a single query, responding in about 0.3 seconds at about four cents per million input tokens, and have been used as classifiers, routers, guardrails, and agent safety reviewers. The authors argue that in safety-sensitive tasks, a decision an attacker can change is a safety failure.
    • Gaps in existing research (authors' view): Xu et al. and Check Point Team use an attacker LLM to generate attack text, and the whole pipeline is slowed by the LLM's generation speed; Wu and Lim search only over 72 fixed template combinations, a limited candidate space. The authors argue none of this work exploits Jev's speed, and all are attacks on a single input.
    • Core hypothesis: classic NLP classifiers serve only one task, whereas a typed decision model defines the task via the developer's question and can serve multiple tasks; the authors therefore hypothesize that a universal suffix found on a single task can transfer to other tasks of the same model.
    • Contributions of the paper: first systematically test common prompt injection (RQ1), then propose a fast black-box query attack that uses no attacker LLM (RQ2), and verify the universality of the suffix (RQ3) and cross-task transfer (RQ4), with experiments on 12 datasets in total.
    • Threat model:
      • Goal: flip Jev's correct and confident decisions, e.g., turning a correct yes into no, or judging unsafe as safe.
      • Knowledge: black-box—the attacker can only read the probabilities Jev returns per query, with no weights, logits, or tokenizer.
      • Capability: can only append a suffix at the end of untrusted content (after a blank line); the suffix must not contain topic words of the question (to ensure the correct label is unchanged), and is at most 32 o200k_base BPE tokens long (64 for trajectory review scenarios).
      • Victim system: applications deploying Jev, where the attacker is an external content author (e.g., email sender, web page author) who cannot change the developer's trusted question.
  2. What related research exists?

    Related work falls into four groups: typed decision model research, adversarial attacks and prompt injection, work that directly attacks Jev, and concurrent Jev attack research.

    Typed decision models

    • Jev is the first such model (the paper cites TypeSafe); recent work uses it as a fast decision layer for agents, an evaluation reviewer, an agent trajectory safety reviewer, and an alignment-failure and prompt-injection detector; an analysis of 2,170 public GitHub projects found applications using it for judging, scoring, action selection, content filtering, and tool selection.
    • Two studies show its answers follow option names rather than option definitions, and another shows misleading option names can open an agent guardrail built on an open-source typed decision model; the paper notes these studies all use benchmark inputs on which the attacker has not adapted to the model.

    Adversarial attacks and prompt injection

    • Black-box text classification attacks search for perturbations with word-level edits; on universal attacks, Wallace et al. found input-agnostic trigger words, and Zou et al.'s jailbreak suffixes are universal and transferable, but Meade et al. found that transfer poor and inconsistent across 13 open-source models.
    • Universal text can also manipulate decision components in LLM applications: inflating LLM reviewer scores, bypassing guard models, and making an LLM router send arbitrary queries to a strong model; the paper notes most of these attacks target a single task, and only two works find cross-task trigger words, both relying on holding the model weights.
    • Adaptive attack studies (Liu et al.'s universal prompt injection, Zhan et al., Nasr et al. bypassing 12 defenses, Hackett et al. bypassing injection detectors) confirm the lesson the authors cite: fixed attack suites measure only resistance to known strategies, and only adaptive attacks measure adversarial robustness.

    Work that directly attacks Jev

    • Wu and Lim recast InjecAgent tool-selection cases into Jev multiple-choice questions and search 72 template combinations; Xu et al. have a language model insert natural sentences; Check Point Team use a multi-turn attacker LLM; the initial JevAdvBench tests a single fixed edit, and Liu shows adding a colon can make Jev accept a wrong answer.
    • The paper notes the adaptive attacks in these works do not exploit Jev's speed and all optimize per input, whereas this paper adds universal suffixes and their cross-dataset, cross-task transfer.

    Concurrent work: the paper notes three differences—the attack text of Wang et al., Lan et al., and the new JevAdvBench may change the correct answer (requiring extra review), whereas this paper's suffixes do not change the correct label by construction; Luo et al., Wang et al., Li et al., etc. use another language model or a fixed set to generate attack text, whereas this paper uses word-level edits plus returned probabilities; none of these works tests suffix transfer across tasks within the same model.

    Baselines and benchmarks: this paper's control arms include common prompt injection strategies (Liu et al.'s naive, context-ignoring, fake-completion, combined, and InjecAgent instructions), seed text without search, no suffix, and neutral comment; the strategy comparison arm is random mutation, hill climbing, and full evolutionary search (Appendices A, E). Evaluation uses 12 datasets (including the R-Judge and ATBench500 trajectory benchmarks).

  3. How does the paper address this problem?

    FlipJev: elite evolutionary search for word-level suffixes, scored only by Jev's returned probabilities; scoring by the average P(no) over 10 documents yields a universal suffix.

    Overall approach: FlipJev does not use an attacker LLM; instead it uses cheap word-level edits to perform elite evolutionary search, spending the entire budget on querying Jev; changing the fitness from a single document to the average P(no) over 10 training documents lets the same algorithm generate a universal suffix.

    Problem setup and constraints

    • Jev input has two parts: a trusted question written by the developer (Noul yes/no questions, Choice option questions, Score rating questions) and untrusted content; the attacker only modifies the content.
    • Suffix constraints: it contains no task-forbidden topic words (e.g., the email task forbids machine, learn, course), is at most 32 tokens counted by o200k_base BPE, and guarantees the correct label is unchanged by construction.
    • Each request asks four questions about the same content: plain/boundary instruction × Noul/Choice; the attack optimizes P(no) under plain Noul, and the other three answers are used only to break ties.

    Evolutionary search (Algorithm 1)

    • The fitness f(x) is the P(no) returned by Jev after appending the suffix; for universal search it is the average P(no) over 10 training documents, with one query per document.
    • It maintains 16 elites; each generation Mutate produces 64 new suffixes from the elites, selecting parents with weights linearly decreasing by rank, applying 1–3 word-level edits (replace, insert, delete, swap, duplicate, insert random string, elite crossover); after 10 generations of stagnation, one additional edit is applied.
    • The vocabulary has about 110 instruction, structure, and framework words (task-forbidden words removed); the email search uses an earlier 192-word vocabulary.
    • Sort key K(x) = (1[f(x)>0.5], f(x)−λ|x|, f̄o(x), −|x|): first by whether it flips, then by fitness minus length penalty (λ=0.0025, a penalty of 0.08 at 32 tokens), then by the mean of the other three answers and length.

    Confirmation and shortening

    • The same request returns probabilities with a standard deviation of 0.03–0.04, so a single query may cross 0.5 by chance; Confirm queries the new best suffix 3 more times and accepts only if at least 2 flip.
    • Shorten deletes words from right to left, keeping a deletion if a single query after deletion still flips; after shortening it goes through Confirm again.
    • The budget B is counted by the number of evaluations of f: 2,000 per document search, 5,000 for universal search (3,000 for trajectory review). The 2,000-query email search takes 1.5–2.6 minutes with 6–8 parallel workers and costs about $0.10.

    Success determination and measurement

    • Flip definition: a correct yes becomes no, or a correct unsafe is judged safe; only correct and confident decisions are attacked (positive examples with P(yes) ≥ 0.8 among clean passes on 9 datasets, unsafe trajectories with score ≥ 2.5 on trajectories).
    • Three measurements (Figure 4): per-document ASR (the proportion of confirmed flips among 20 targets, verified for repeatability by 10 more queries after confirmation); universal ASR (the flip proportion of one suffix found on 10 training documents over 30 held-out documents, taking the majority of 10 queries per document); transfer ASR (the flip proportion of one dataset's universal suffix on the first 20 held-out documents of another dataset, 3 queries each).
    • Controls: no suffix, neutral comment; for the trajectory setting, additionally equal-length neutral padding and seed text. Ratios use exact binomial 95% intervals, and score drops use bootstrap intervals.
  4. What experiments did the paper conduct?

    Four RQs across 12 datasets: common injections have zero flips; query attacks flip 58/180 targets; universal suffixes flip 43% of held-out documents and transfer across tasks.

    Experimental Setup

    • Target model: Jev, accessed via the OpenRouter API, snapshot typesafe/jev-1.13-20260917, returned probabilities rounded to 0.01.
    • Datasets: 12 — email understanding, TREC-QC, AG News, DBpedia-14, Yahoo! Answers, patent abstracts, arXiv abstracts, 20 Newsgroups, LexGLUE ECtHR, arXiv full text (documents all 15–2,831 tokens long), plus the agent trajectory safety benchmarks R-Judge and ATBench500; IMDB is excluded from the total and reported separately because its sentiment can be changed by appended text.
    • Targets and splits: 9 datasets each take 20 attack targets, 10 training documents, 30 held-out documents (24 for 20NG), the three disjoint; the email task has 28 searches (6 seeds × 9 yes emails).
    • Metrics: flip rate (ASR) and average drop in trajectory safety score; intervals are exact binomial 95% CI or bootstrap CI.
    • Controls: common prompt injection (5 strategies), seed text without search, no suffix, neutral comment; trajectory scenarios add equal-length neutral padding.
    • Repetitions and queries: held-out takes the majority of 10 queries per document, transfer 3 per document; the paper reports no LLM judge (decisions are based on returned probability thresholds, no LLM judge).

    Main Results

    According to Table 1 (proportion of correct and confident decisions flipped; searches per document: 28 searches for email, 20 targets each for the others; universal suffix: searched from 10 training documents, used on 30 held-out, 24 for 20NG):

    Swipe horizontally to read the table

    MethodEmailTRECAGDBPYahooPatentarXivA20NGECtHRarXivFR-JudgeATBench
    Common injection0.000.000.000.000.000.000.000.000.000.000.000.00
    Seed text without search0.070.000.000.000.000.000.000.000.000.000.000.03
    Per-document search0.390.650.300.050.450.300.400.550.150.050.050.15
    Universal suffix0.470.570.100.630.330.670.100.420.570.500.030.30
    • The authors state that Jev resists common prompt injection: RQ1 found 0 strict flips in 10,240 paired tests under 4 question configurations on the email task, and even a forged Jev JSON answer left plain-Noul P(correct) within 0.004 of the control; the 5 strategies likewise produced zero flips on held-out inputs of the other 11 datasets (1,620 pairs total).
    • The authors state that the black-box query attack is effective: on email, 11 of 28 searches confirmed a flip (39%, 95% CI 22–59%), median 1,287 queries; on 9 datasets, 58/180 targets flipped (32%, 95% CI 26–40%), suffixes 5–29 tokens (median 11). Figure 2 shows most flips occur after 1,000 queries and the curve is still rising at the end of the budget; the authors state these rates are lower bounds for a larger budget.
    • The authors state that suffixes are highly universal: on 9 datasets the universal suffix flipped 114/264 held-out documents (43%, 95% CI 37–49%), versus 0/264 for no suffix and neutral comment; on 4 of these datasets the universal suffix's flip rate was higher than per-document search.

    Cross-task Transfer (RQ4)

    • What was tested: each dataset's universal suffix was applied to the first 20 held-out documents of the other 8 datasets (3 queries each, no further search), 72 ordered pairs in total; between trajectory benchmarks, the ATBench suffix was applied to 20 held-out R-Judge trajectories.
    • Results: across the 72 pairs the average flip was 18%, 21 pairs ≥ 20%, 11 pairs ≥ 50% (Figure 3). The arXiv full-text suffix (question about computer science) flipped 90–100% on AG News, DBpedia, arXiv abstracts, patent, and 20NG, and 45% on ECtHR; 20NG as a target was most susceptible to transfer, with foreign suffixes flipping 45% on average. The ATBench suffix lowered held-out R-Judge scores by 0.38 on average, comparable to R-Judge's own suffix.
    • Authors' interpretation: applications of different tasks can share one typed decision model, so an attacker can find a suffix on a queryable task and then attack other applications with zero queries; the authors consider this a risk that classic NLP classifiers do not have.

    Trajectory Safety Review and Mechanistic Analysis

    • Trajectory review (Appendix H, Table 13): the best per-document suffix lowered R-Judge scores by 0.42 and ATBench by 0.49 on average over 10 fresh queries (covering 0.46 and 0.53 of the distance to the decision threshold, respectively); the ATBench universal suffix (a passage of review-record-style text) flipped 9/30 held-out unsafe trajectories (30%, 95% CI 15–49%) with an average score drop of 0.65, and the R-Judge universal suffix lowered scores by 0.37. Figure 9 shows equal-length neutral padding changes the mean score by at most 0.01; the authors state the effect comes from what the suffix says, not its length.
    • Mechanistic analysis (Appendix G, Tables 8–10): the authors compare two explanations, "instruction following" and "surface pattern". After ablation all variants' flip rates drop below 4% (swapping label words 0.03, removing question 0.04, removing label words/shuffling/label words alone/seed/no suffix all 0.00); the authors state the effect depends on the specific order of specific words; the boundary instruction does not weaken the effect (155/264 flips, higher than 122/264 without boundary), and on the 5 datasets that retain the full seed frame it instead pushes flips to 29–30/30; the authors attribute this to the boundary instruction sharing words such as question, label, and output with the suffix; the Noul suffix does not affect Choice answers on 150 held-out documents, and a suffix searched against Choice flips Choice 22/60 on DBpedia and patent while Noul flips only 2/60. The authors therefore conclude the suffix is not an instruction but a surface pattern, consistent with the existing finding that the typed decision model follows option names rather than definitions.

    Other Ablations and Analyses

    • Strategy comparison (Appendix E, 3 long emails, claim seed, 1,500 queries): evolutionary search and hill climbing each flipped 1 email, random mutation without feedback reached a maximum P(no) of only 0.07.
    • Seed effect (Appendix E): command seed succeeded 1 of 8 searches, claim 4/8, question 3/3, confidence 2/3, computed 0/3.
    • Cross-email transfer (Appendix E): 6 suffixes flipped 12–21 of 49 non-source emails; by type, long natural emails 74%, padded long emails 75%, short hard emails 26%, short easy emails 12%.
    • Email search answer distribution (Appendix E): of 47,173 responses, plain Noul had 126 with P(wrong)>0.5, plain Choice only 4.
    • IMDB (Appendix C): per-document search flipped 13/20, universal suffix flipped 13/30; not counted in the total because appended text can change review sentiment.
    • Negative results: universal search on AG News and arXiv abstracts did not push the training mean above 0.5 within 5,000 candidates (marked † in Tables 6, 7); per-document search on DBpedia and arXiv full text flipped only 5%; the R-Judge universal suffix flipped only 3%; on patent, search against Choice did not push the training mean above the 0.50 threshold after 5,000 candidates (Table 12).
  5. What are some points that can be further explored?

    The authors list two limitations: semantic-change attacks are out of scope, and the search components could be further streamlined; the experiments cover only Jev and English public datasets.

    Limitations and Future Directions Noted by the Authors

    • Attacks that change the semantics of the content belong to another threat model and are out of scope for this paper (Limitations section).
    • The authors state that if the mutation operators or other components were more refined, the attack might find flips with fewer queries; but the current attack already needs only about two minutes per input and can find universal suffixes that transfer across tasks, so they leave such streamlining as future work (Limitations section).

    Scope of Experimental Coverage

    • The only model tested is Jev (via OpenRouter, snapshot typesafe/jev-1.13-20260917); a paper footnote states that at the time of the study Jev was the only closed-source commercial typed decision model.
    • The data are 12 English public datasets and benchmarks (including the R-Judge and ATBench500 trajectory benchmarks), with 20 targets, 10 training, and 30 held-out per dataset (24 for 20NG); IMDB is reported separately and not counted in the total.
    • The attack surface covers only appending a suffix at the end of untrusted content; injection in the developer's trusted question is measured only as a control in Appendix D (a request placed in the trusted question lowers P(correct) but did not flip decisions).
    • The budget is 2,000 queries per document and 5,000 for universal (3,000 for trajectories); Figure 2 shows the flip curve is still rising at the end of the budget, and the authors state the reported rates are lower bounds for a larger budget.
    • The paper reports no evaluation of any defense against this attack; the ethics statement says it has been disclosed to TypeSafe through the vendor's official channel, and that code, suffixes, and result files are planned for release.
  6. Summarize the main content of the paper

    A fast black-box attack on Jev: a universal suffix flips 43% of unseen documents and transfers across tasks; the authors call for evaluations to include strict adaptive attacks.

    A red-teaming study of typed decision models: the authors propose FlipJev, a fast black-box query attack, against the representative model Jev, and systematically measure its adversarial robustness.

    • Problem: Jev returns per-option probabilities in a single query and has been used as a classifier, guardrail, and agent safety reviewer; the authors argue existing adaptive attacks are slowed by the attacker LLM or limited by template sets, and no one has searched for universal adversarial examples that span inputs and tasks.
    • Method: no attacker LLM; elite evolutionary search with word-level edits optimizes a short suffix (≤32 BPE tokens, containing no topic words), scored only by Jev's returned probabilities; scoring by the average P(no) over 10 training documents yields a universal suffix, with 2,000 queries taking about two minutes and about $0.10.
    • Common injection ineffective: 5 common prompt injection strategies produced 0 strict flips in 10,240 paired tests on the email task, and also zero flips on held-out inputs of the other 11 datasets (Tables 1, 4).
    • Adaptive attack effective: per-document search flipped 58/180 targets on 9 datasets (32%), and 11/28 searches succeeded on email; one universal suffix per dataset flipped 114/264 held-out documents (43%), versus 0 for neutral comment.
    • Cross-task transfer: a suffix found on arXiv full text flipped 90–100% of documents on 5 other datasets with zero queries (Figure 3); the ATBench universal suffix flipped 30% of held-out unsafe trajectories, which the authors state exposes the practical risk of Jev as a safety guardrail.
    • Authors' conclusion: mechanistic analysis shows the suffix is a surface pattern rather than an instruction (flip rate ≤4% after ablation, boundary instruction does not protect); the authors argue that evaluation of typed decision models must include strict adaptive attacks, and their robustness still needs to be evaluated and strengthened.

Related papers

Read originalarxiv.org