LLM trojans that increase response length and cost

When the trigger appears, lengthen the instruction-tuned model's responses, thereby raising costs billed by output tokens.

• Scenario: The authors state that GenAI SaaS is often billed by input and output tokens, and give the example that some providers' unit price for output tokens is usually five times that of input, with prefill discounts of up to 90%. Overly long responses consume decoding computation and memory; the authors believe frontier models should aim for high quality without being overly long, otherwise long-term economic viability and adoption are affected.
• Existing gaps: The authors state that existing backdoors mostly target classification, refusal, factuality, irrelevant answers, or instruction redirection. This paper targets response length and makes it easy for honest users to trigger unintentionally. At post-training certification time, the training set is usually unavailable; on LLMs, words with intrinsic features of the target class can also push the true trigger out of the high-score list.
• Approach: The authors propose a length backdoor, divided into a neutral prefix trigger and a synonymous trigger; then they extend class subspace orthogonalization (CSO) to detection without access to the fine-tuning set, trigger inversion, and unlearning mitigation.
• Threat model:
  • Goal: When the trigger is present, responses are far longer than requested; when the trigger is absent, behave like a clean model, so that validation on clean data does not expose the backdoor; summaries after the trigger remain fluent and faithful, so that neither users nor quality monitoring easily flag them.
  • Knowledge: Cannot access the weights, the training process, or the deployed system. The paper does not describe the attack as white-box or black-box.
  • Capability: Control a small part of the instruction-tuning data, e.g., a public instruction dataset or outsourced fine-tuning. Dirty-label poisoning: modify prompts asking for short summaries and pair them with long reference summaries instead of short ones. Poison only 2% of samples requesting short summaries; verbose samples are unchanged.
  • Victim: The instruction-tuned summarization model. One type of trigger prefixes a neutral phrase to the short instruction; another type replaces the short instruction with another wording that asks for brevity.

Grouped according to Section 2 and the introduction.

Image classification backdoors

• BadNets (Gu et al., 2019): implant a trigger in the training data so that the classifier outputs the attacker-specified label when the trigger appears. The authors attribute early image classification backdoors to Gu et al. (2019) and Wang et al. (2019).
• Post-training image detection often assumes how the trigger is embedded: patch replacement in Wang et al. (2019), additive perturbation in Xiang et al. (2022).

Goals of LLM backdoors

• Zhao et al. (2024) survey recent LLM backdoor attacks and defenses. The goals listed by the authors include: text classification, inducing refusals that violate safety alignment, changing factuality, outputting specified irrelevant content, and instruction following (with two instructions, the trigger makes the model execute the other one).
• The authors distinguish colluding with the poisoner, users who actively add the trigger, and cases relying on honest users triggering unintentionally, and place this paper in the last category: the trigger looks consistent with "answer briefly" and extends to synonymous phrasings.

Post-training defenses and CSO

• When the training set is available, Chen et al. (2019) use activation clustering and Tran et al. (2018) use spectral signatures to detect poisoned samples before retraining. The authors state that proprietary or frontier models are often already trained and the training set is unavailable, which is post-training certification; usually there are only a small number of clean samples without trigger cues.
• MM-BD (Wang et al., 2024) and UNICORN (Wang et al., 2023) do not assume a trigger embedding mechanism. Most classifier detection solves an optimization for each class, and if the statistic of a class is clearly an outlier, it is judged poisoned.
• CSO (Yang et al., 2026) adds a class subspace orthogonal penalty to the detection objective: suppress class-intrinsic features and leave the backdoor contribution. CSO-LLM (Li et al., 2026) applies it to discrete, variable-length outputs and addresses the problem that intrinsic-feature words occupy high scores and the true neutral trigger ranks very low. The authors state that this penalty acts as an implicit blacklist, and that no prior work accurately inverts trigger phrases on LLMs; hyperparameters use unsupervised red-teaming. ONION (Qi et al., 2021) is used to illustrate that words clearly inconsistent with the context are easy to find and unsuitable as triggers.

Billing background and experimental baselines

• The introduction discusses billing by input tokens, by output tokens, and hybrid subscriptions, as well as FrugalGPT (Chen et al., 2023), in which users choose among models to reduce cost. These are not the attack or defense baselines of this paper.
• Baselines: The attack control is a clean fine-tuned model on the same data with a poisoning ratio of 0%; the defense control is ML, which only maximizes length deviation without CSO. Benchmarks are Multi-LexSum and CNN/DailyMail.

The authors position this paper as: the attack targets response length rather than content, and synonymous triggers make unintentional triggering easy; the defense applies CSO-LLM to detection, inversion, and unlearning of length backdoors, without access to the possibly poisoned fine-tuning set.

Plant a length backdoor in summarization instruction tuning, then use a length objective with a CSO penalty for post-training detection, discrete inversion and lightweight unlearning.

Poisoning setup

• The document is x, the instruction is I, the response is M(x⊕I). Each document has a short reference rshort and a long reference rlong. Clean samples are brief instructions paired with rshort and verbose instructions paired with rlong. In the main text the brief instruction is written as asking for a tersely summary, and the verbose instruction asks for a verbosely summary.
• Poisoning ratio p=2%, only brief samples are changed, verbose samples are untouched. The neutral trigger prepends the phrase to the brief instruction, and the supervision is changed to rlong. The synonymous trigger replaces the brief instruction with another way of asking for brevity, also paired with rlong. The control is clean fine-tuning with p=0%.
• The neutral phrases in the experiments are seriously, and the three-word phrase Tell me seriously. The synonymous trigger instruction in the experiments uses Briefly. The main text and appendix give the full instruction sentences.
• The attack succeeds only if three goals hold simultaneously: longer under trigger; like a clean model without trigger; triggered summaries still fluent and faithful.

How length and faithfulness are measured

• The inflation rate divides the number of generated tokens by the same model under the test instruction by its own length under the brief instruction: IR(I′)=100%× E[|M(x⊕ I′)|]/E[|M(x⊕ I_terse)]|. IR around 100% means the instruction does not change length. The denominator is the model's own brief baseline, so it measures the effect of the instruction, not how long the model is by nature. The same phrase is also tested on clean models, to separate the backdoor from the phrase itself.
• Whether the absence of a trigger exposes the model is measured by SR, the length ratio between the poisoned model and the clean model under the brief instruction (Appendix Equation 7). SR around 100% means clean instructions show no sign of poisoning.
• Faithfulness uses ROUGE-Lsum, ROUGE-2, BERTScore: brief outputs against rshort, verbose and triggered outputs against rlong. There are also length-independent LLM judgments Rel, Cons (1–5); Cons is used to see whether lengthening relies on confabulation padding.

Data, models and fine-tuning

• Multi-LexSum case files are too long, so inputs use the dataset's long summaries, with tiny and short as rshort and rlong respectively, keeping only cases with all three summaries. CNN/DailyMail uses the article as input and highlight sentences as rshort, with rlong produced by gemma-4-E2B-it expanding the reference into a long summary. Lengths are in Table 6.
• Four open-source models are LoRA fine-tuned with frozen base weights and prompt tokens excluded from the loss: Qwen3-0.6B, Qwen3-8B, Gemma3-1B, Gemma3-12B. Appendix A.2: r=16, α=32, dropout 0.05, 2 epochs, AdamW, cosine schedule, warmup ratio 0.03. The two datasets differ in learning rate, batch size, maximum input and maximum new tokens at test time, see Table 7.

Detection, inversion and mitigation

• The defender has a small clean set, split into brief and verbose, containing no trigger clues, and does not access the fine-tuning set. Candidate z replaces the brief adverb under the synonymous hypothesis, and is prepended under the neutral hypothesis; the main text writes the objective for the synonymous case, and for the neutral case only changes "replace" to "prepend".
• The nominal objective is the average length deviation: generated length under the candidate trigger minus the brief reference length. Maximizing only this yields explicitly length-seeking words such as expand, detailed. The authors state such words also lengthen responses on clean models, and their intent is obvious, unlike a trigger an attacker would choose.
• The CSO penalty is the cosine similarity of the hidden states of internal layers at the last prompt token along the "brief sample to verbose sample" direction. What is maximized is J_{t→ v}(z)=Δ L_{t→ v}(z)−λ CSO_{t→ v}(z), λ>0. The layer and λ do not rely on known clean/poisoned reference models, but on red-teaming.
• The search greedily accumulates by token, keeping only the top N=10 by score at each step, stopping when the relative score change is less than δ=0.1, and returning z*. If the length deviation of this z* exceeds threshold θ, it is judged a length backdoor. L=5 clean variants and L′=5 poisoned variants are copied from the model under test (p′=0.02, known two-token trigger), and the layer, λ and θ are jointly selected on them. Appendix A.7 lists the selected values.
• If judged poisoned, z* is inserted into clean samples with reference responses unchanged, and lightweight fine-tuning is done on Nt=Nv=150. Re-testing IR still uses the real trigger, not z*.

Experimental setup

• Models under test: Qwen3-0.6B, Qwen3-8B, Gemma3-1B, Gemma3-12B, all LoRA fine-tuned summarization models. CNN/DailyMail long references are expanded by gemma-4-E2B-it. The judge model is Meta-Llama-3.1-8B-Instruct, which the authors state avoids the Qwen family to reduce self-evaluation preference.
• Data: Multi-LexSum and CNN/DailyMail. Table 6: the former counts only cases with all three summaries, train 1,129/3,177, test 312/908; the latter train 287,113, test 11,490. Evaluation uses 200 held-out documents.
• Protocol: poisoning ratio 2%. Greedy decoding; Multi-LexSum at most 512 new tokens, CNN/DailyMail at most 256. IR is the mean±standard deviation over 10 random seeds; quality scores are means over 200 documents. Detection is counted over 10 independently poisoned or clean fine-tuned models.
• Metrics: IR (Equation 1), SR (Equation 7), R-L, R-2, BERTScore, and Rel, Cons (1–5). The judge does not sample; it reads the score from the next-token distribution over 1 to 5, and is instructed to ignore length. Appendix A.6 gives the prompts and rubric. ROUGE uses Porter stemming, R-L is rougeLsum; BERTScore uses roberta-large with baseline rescaling.
• Defense: The control is ML, which removes CSO and only maximizes length deviation. Candidate pool N=10. Mitigation uses each model's own inverted trigger, Nt=Nv=150. The layer, λ, θ selected by red-teaming are in Table 16: θ=220 on Multi-LexSum, θ=170 on CNN/DM.

Main results

According to Table 1, IR of triggered prompts (%, mean over 10 seeds; standard deviations in that table). MLS is Multi-LexSum.

Swipe horizontally to read the table

Model	MLS seriously	MLS Tell me seriously	CNN/DM seriously	CNN/DM Tell me seriously
Qwen3-0.6B	672.7	675.4	191.6	228.8
Qwen3-8B	681.4	726.8	164.3	219.5
Gemma3-1B	618.4	671.3	187.0	191.5
Gemma3-12B	642.5	701.5	212.6	197.6

• The authors state that on clean models this trigger changes length by less than 7%. The none row of Table 1 ranges from 99.7±0.4 (Qwen3-0.6B, CNN/DailyMail) to 106.4±3.1 (Gemma3-12B, Multi-LexSum). The authors state that after poisoning Multi-LexSum inflates 6.2–7.3× and CNN/DailyMail 1.6–2.3×, and that this roughly follows the ratio of the two reference lengths: about five on the former and about three on the latter in Table 6.
• The authors state triggered summaries remain faithful. In Table 10, Qwen3-0.6B, Multi-LexSum, τ=seriously: triggered R-L 0.465, R-2 0.311, BS 0.356, Rel 3.879, Cons 4.363, with the verbose ceiling at 0.447, 0.303, 0.335, 3.909, 4.344.
• The authors state that across four models, two datasets and two neutral triggers, triggered prompts have Rel and Cons higher than brief outputs (Cons higher by 0.01 to 0.27), and on Multi-LexSum triggered R-L is no more than 0.08 below the verbose ceiling. Other models are in Tables 11–13. On CNN/DailyMail, where inflation is smaller, reference-based scores fall between brief and verbose: in Table 10, Qwen3-0.6B, Tell me seriously has triggered R-L of 0.367, brief 0.329, verbose 0.552; the authors state Cons is not lower than the brief baseline, so the extra tokens are not hallucination padding.

Synonymous triggers and trigger-free stealth

• Synonymous poisoning is still 2%, replacing brief samples with another brief-seeking instruction paired with long references. Section 3.5 discusses Qwen3-0.6B and Qwen3-8B on Multi-LexSum. Tests keep the sentence pattern and only swap the brief adverb, compared against p=0.
• The authors state that in Figure 1 (0.6B) and Figure 2 (8B), clean models already show length inflation on the synonymous instruction briefly, and poisoned models show more, with lengths approaching the verbose instruction, so honest users can easily trigger it unintentionally. The captions only label left as clean and right as 2% poisoned; the text gives no per-bar values.
• Table 14 is titled Multi-LexSum, IR of Iτ: Qwen3-0.6B 679.18±30.91, Qwen3-8B 712.52±31.84, Gemma3-1B 192.84±16.03, Gemma3-12B 197.27±15.68. The Section 3.5 main text discusses only the first two Qwen models. SR in Table 9: CNN/DailyMail, Tell me seriously, Gemma3-1B is 95.03; CNN/DailyMail, seriously, Qwen3-8B is 111.81; the authors state that relative to clean models some rise slightly and some fall slightly. The table note says synonymous triggers were not evaluated on Gemma3-12B; the extracted cell value for that entry does not match the table note, so it is not used.

Detection, inversion and mitigation

• Table 2 is the number judged poisoned out of 10 models. Where column alignment is clear, CSO on Multi-LexSum: Qwen3-0.6B is 10 for all three triggers, 2 for clean; Qwen3-8B is 9, 10, 9, clean 2; Gemma3-1B is 8, 10, 10, clean 2. CSO on CNN/DM: Qwen3-8B Tell me seriously is 8, seriously 10, clean 0; Qwen3-0.6B and Gemma3-1B clean models are both 3. The authors state ML's true positives are comparable to or slightly higher than CSO, and clean-model false positives are higher, e.g. on CNN/DM Qwen3-0.6B and Gemma3-1B are 10, Qwen3-8B is 7. The Briefly column for Gemma3-12B is 100 in the extracted text, inconsistent with "10 models", so it is not used.
• Table 3: Qwen3-0.6B, CNN/DM, Tell me seriously, λ=500. ML's top-10 includes explicitly length-seeking words such as Expanded (ΔL 240.2), extensively (236.1). CSO ranks seriously 7th (ΔL 221.2, ML rank 150). The authors state CSO pushes most such false triggers from the top-10 down to the hundreds.
• Table 4: single word seriously, CSO and ML are both 10/10 across four models and two datasets. Three-word trigger CSO is 24/30 to 30/30 (lowest and highest of the eight cells), ML is 7/30 to 19/30. Briefly only on Multi-LexSum: CSO is 14/20, 19/20, 15/20, 17/20, ML is 0/20 for all four models. The paper does not explain how the denominators 10, 20, 30 correspond.
• Table 5: unlearning uses the inverted trigger, IR uses the real trigger, mean over 10 poisoned models. The authors state that after mitigation IR falls between 100.2% and 113.5%, versus 99.7% to 106.4% for clean models in Table 1, and the trigger no longer inflates responses. For example Multi-LexSum, seriously, Gemma3-12B goes from 642.5 to 100.2; CNN/DailyMail, Tell me seriously, Gemma3-12B goes from 197.6 to 113.5. The authors state no fine-tuning set is needed, only Nt=Nv=150 clean samples.

Other ablations and analysis

• Table 15: four clean models, Multi-LexSum, replacing the brief instruction word with 10 synonymous rewrites of Verbosely, IR is 552.6%–833.3% (mean over 200 documents). The authors use this to state that explicit length-seeking needs no poisoning, and these words are false triggers during inversion.
• Table 16: CSO layers are 2 to 12, λ is 400, 500 or 600; θ is 220 on Multi-LexSum and 170 on CNN/DM.
• Appendix A.4: single-word scan time T is 7 to 20 minutes on a single A100 (Table 8). With N=10, a three-word trigger is about 51T; adding 10 red-team variants, defending one backdoored model is about 261T, about 26T on ten GPUs. The scan is limited to English tokens, about 41,000 for Qwen3 and about 53,000 for Gemma3.
• Appendix A.7: red-teaming uses a two-token non-descriptive trigger, p′=0.02. The authors state that with λ from 0 to 600, the first step always keeps the first token in N=10; scanning λ from 300 to 800 mainly affects whether the second step can recover the whole trigger.

Limitations and future directions noted by the authors

• The conclusion (Section 5) gives only one future direction: exploring reducing the complexity of the discrete cumulative search used for trigger inversion. The paper has no separate Limitations section and does not write other phenomena as limitations or future work.

Scope of experiments

• The models tested are Qwen3-0.6B, Qwen3-8B, Gemma3-1B, and Gemma3-12B, 4 in total, all LoRA instruction-tuned summarization models (Section 3.3).
• The datasets are Multi-LexSum and CNN/DailyMail. The synonymous-trigger body and Figures 1–2 are Qwen3-0.6B and Qwen3-8B on Multi-LexSum (Section 3.5). The Table 9 caption states that synonymous triggers are evaluated only on Multi-LexSum.
• Attack metrics include IR, SR, R-L, R-2, BERTScore, Rel, and Cons. IR is over 200 held-out documents and 10 seeds; quality scores are the mean over these 200 documents (Section 3.3, Appendix A.5).
• Defenses include CSO and ML detection (10 fine-tuned models each), trigger inversion, and unlearning with the inverted trigger and Nt=Nv=150 (Sections 4.3, 4.5). The inversion scan is limited to English tokens (Appendix A.4).
• The judge model is Meta-Llama-3.1-8B-Instruct. The paper does not report the agreement between this judge and human ratings, nor does it explain how the inversion denominators 20 and 30 in Table 4 correspond to the 10 models used for detection. Search overhead is given as T in Appendix A.4 and as approximately 51T and 261T.

The authors study a length backdoor that inflates output token cost, and give a post-training defense that does not access the fine-tuning set.

• Problem: When billing is by output token, overly long responses increase user cost. The authors want an instruction-tuned summarization model to become longer under a trigger, to behave like a clean model without the trigger, to keep summaries faithful, and honest users may unintentionally trigger it through synonymous wording.
• Attack: Dirty-label poisoning of 2% of samples requesting brief summaries, re-paired with long references; verbose samples are unchanged. The trigger is either a prepended neutral phrase seriously / Tell me seriously, or replacing the brief instruction with Briefly. The attacker cannot access weights, the training process, or the deployed system.
• Defense: On a small set of clean samples, maximize length deviation, subtract the CSO penalty, to suppress explicit length-seeking false triggers; greedily accumulate to invert the trigger, and if it exceeds the red-teaming threshold, judge poisoning, then insert the inverted trigger into 150 clean samples for lightweight unlearning.
• Attack numbers: In Table 1, Tell me seriously, Multi-LexSum, Qwen3-8B has IR of 726.8%; same trigger, CNN/DailyMail, Qwen3-0.6B is 228.8%. The authors state clean models change by less than 7%, and after poisoning the two datasets inflate by 6.2–7.3× and 1.6–2.3× respectively. In Table 14, synonymous trigger, Qwen3-8B, Multi-LexSum has IR of 712.52%. The authors state Rel and Cons are not lower than brief outputs.
• Defense numbers: In Table 4, inversion of seriously, CSO and ML are both 10/10; on Briefly, CSO is 14/20 to 19/20, ML is 0/20. In Table 2, ML has higher false positives on clean models, with many cells at 7 to 10. The authors state that after mitigation IR falls between 100.2% and 113.5% (Table 5).
• Authors' conclusion: The authors state synonymous triggers generalize to other ways of asking for brevity, giving examples in the conclusion of succinctly, compactly, shortly, and thus are easily unintentionally triggered by honest users; CSO can detect, invert and unlearn this backdoor on summarization tasks. Future work can reduce the complexity of the discrete accumulation search.