CacheTrap: Researchers propose a gray-box Trojan attack against LLM KV caches

Without modifying inputs and weights, and without gradients, how can a single transient trigger make an LLM output an attacker-specified class.

• Scenario: The authors argue that LLMs have entered critical applications, and Trojans can embed triggerable malicious behavior. The conventional insertion methods are training or data poisoning, or injecting memory faults at runtime.
• Limitations of existing work: The authors divide existing work into Type-I and Type-II and compare them in Table I. Type-I modifies data and weights during training or fine-tuning, requires specific input triggers at runtime, often harms benign utility, and leaves traces in inputs and weights. Type-II perturbs weights at deployment, destroying utility and being traceable by integrity checks. Both types assume access to weights; the authors state that real deployments often expose the architecture but not the weights or training data.
• Core observation: Without modifying weights and inputs, the authors turn to the KV cache. It stores intermediate attention states used for decoding, is tied to token positions, resides in GPU memory or can be swapped off-chip, and thus can be targeted by fault injection; when the trigger is absent, weights and inputs remain unchanged.
• Threat model:
  • Goal: steer the output toward a specified class during inference; maintain benign performance when the trigger is not applied.
  • Knowledge: described by the authors as gray-box. Only the public architecture is available; the victim's weights, training data, domain distribution, and gradients are unavailable.
  • Capability: use a public model of the same architecture and public data to offline locate vulnerable positions in the KV cache, then use fault injection to flip the bit at that location. Examples in the paper are GPUHammer and Rowhammer; knowing the target index relies on the KV cache inversion cited by the authors.
  • Victim system: the victim fine-tunes a public LLM on specific data and protects the fine-tuned weights; the architecture remains public. The attack targets the KV cache at inference time.
• Contribution: The authors state that CacheTrap is the first gray-box Trojan targeting the KV cache, using a single-bit flip as a transient trigger, with a search that does not use data, weights, or gradients.

The paper has no separate Related Work section; related discussion is mainly in the introduction, Table I, and Section VI.

Training-time backdoors (Type-I)

• Modify inputs and weights: BadEdit (2024), Trojaning language models (Zhang et al., 2021), Spinning language models (2022), PPT (2022), TrojFSP (2024), i.e., [5]–[9] in the paper. They place a trigger in the input and optimize weights so the trigger corresponds to attacker-defined behavior. The authors state this requires training facilities or training data, needs specific inputs at runtime, often reduces benign utility, and leaves traces in weights and inputs.
• Corresponding defenses: few-shot backdoor defense by Xi et al. (2023), data-free removal based on channel Lipschitzness by Zheng et al. (2022), and BAIT (Shen et al., 2025). The authors state in Section VI that these methods assume the Trojan is hidden in suspicious input triggers or persistent weight poisoning.

Deployment-time weight perturbation (Type-II)

• Weight bit flips: GenBFA (2024), SBFA (2025), SilentStriker (2025), FlipLLM (2025). They offline locate sensitive parameters and compute the magnitude and direction of perturbation, then flip weight bits at deployment using fault injection such as Rowhammer. The authors state that after injection, utility is destroyed and traces remain in performance and weights.
• Integrity-type defenses: RADAR (2021), HASHTAG (2021), output code matching (2022), DNN-Defender (2024). Table I lists them as Type-II defenses; the paper does not rerun these defenses.

KV cache and fault injection

• Cache inversion: Wu et al. (2025) discuss prompt leakage under multi-tenant KV-cache sharing; Shadow in the Cache (Luo et al., 2025) discusses KV cache privacy risks. The authors use these to show that the cache can be inverted, thereby knowing the target index. The paper does not use them as baselines for attack effectiveness.
• GPU fault injection: GPUHammer (Lin et al., 2025) and GPUBreach (Lin et al., 2026). The authors state the former can perform precise bit flips on GDDR6 even with target row refresh and ECC enabled; the latter is described by the authors as enabling GPU-side privilege escalation, bypassing memory isolation and gaining VRAM write capability. These are citations for the execution method, not ASR comparisons.

Baselines and data

• Experimental comparison: The paper does not report ASR for other attacks. The utility comparison is against the uncorrupted model: the trigger-free Post-Attack Acc in Table II is described by the authors as identical to the uncorrupted baseline. Task data are ARC-Easy, ARC-Challenge, TREC, OpenBookQA; LSS uses WikiText-2.

The authors place the paper in the gray-box setting: in Table I, CacheTrap requires no gradients, weights, or data, utility degradation is no, the defense column is Not Available, and the attack vector is K–V. The authors state this relaxes prior assumptions about weights and leaves no conventional traces in inputs or weights.

CacheTrap performs a forward-only offline search on a public model with the same architecture in the attacker's own space, records the KV cache coordinates, and then flips the corresponding bit during victim inference.

Problem Setting

• The victim is an LLM fine-tuned on specific data with protected weights. The attacker does not have the weights, task data, or gradients, only the public architecture.
• The search is done on a public model with the same architecture; calibration data is a small sample that is public and mismatched to the victim's task.
• The budget is a single transient bit flip at inference time. The goal is to push the prediction toward a specified class regardless of the input.
• The transfer question the authors address is whether positions found on a public model transfer to the corresponding task-fine-tuned model.

Layer Sensitivity Score (LSS)

• The authors first narrow down the layer range, based on outlier activation in LLMs: certain layers disproportionately amplify activations, and KV bit errors in these layers have a greater impact on the output.
• The score for layer ℓ is L_ℓ=|σ(h_ℓ)−σ(h_ℓ−1)|. σ is the standard deviation across tokens and samples; a large difference indicates that the layer causes a stronger distribution shift.
• Keep the top-scoring layers. The authors state that LSS depends only on general activation statistics and is task-independent. In implementation, LSS uses WikiText-2. The paper does not specify the number of retained layers.

Cache Vulnerability Score (CVS) and Token Position

• The K and V of seen tokens are stored in the cache, and subsequent suffixes are decoded using these prefixes. The authors by default corrupt the value vector of the last prefix token, because it encodes almost the entire input context and directly affects the next decoding step.
• Within sensitive layers, compute C_ℓ,h,j=|[V_ℓ,h(T,j)]_{x∈ Dcalib}|₂ for head-channel (h, j). The authors assume that high-magnitude channels contribute more to attention readout.
• For each layer, take the top-p coordinates with the largest C. The paper does not specify p, only that offline construction can be done in a few minutes.
• The authors state that other token positions are also allowed; by default, the last prefix token is used, and the choice is discussed in Section V-B6 with offset experiments.

Class-wise Search and Success Determination

• For each candidate coordinate, make a single-bit modification on the stored value of the selected token, allowing only one bit flip per round. The authors state that flipping bits that affect the value's magnitude has a greater downstream effect; the software search part does not specify which bit.
• Compute ASR by class on the public calibration set: after flipping the coordinate, the proportion of samples predicted as class c. Starting from earlier sensitive layers, for each c select the coordinate with the largest ASR among the top-p.
• The stopping condition is that the ASR for each class is not below a preset threshold τ. The paper does not give τ. The obtained coordinates are used in evaluation.
• On the victim test set, to predict a sample as c, flip one bit at the corresponding coordinate. ASR is the proportion of test samples predicted as c. Without the trigger, no weight modification is made, and the authors state that benign accuracy remains unchanged.

Experimental Setup

• Target models: LLaMA-2-7B-Chat, LLaMA-3.1-8B-Instruct, Mistral-7B-Instruct-v0.1, Qwen2.5-3B-Instruct, DeepSeek-R1-Distill-Qwen-7B. Each is fine-tuned and used as a victim. Table II abbreviates them as LLaMA-2-7B, LLaMA-3.1-8B, Mistral-7B, Qwen-2.5-3B, DeepSeek-7B.
• Search models: Public models with the same architecture are used for offline search. Tables III and IV specify the public model as pre-trained LLaMA-3.1-8B, and the victims are the corresponding fine-tuned models.
• Data: Victim tasks are ARC-Easy, ARC-Challenge, TREC, OpenBookQA. LSS uses WikiText-2. CVS uses 100 mismatched samples: when attacking OpenBookQA, TREC, or ARC-Challenge, ARC-Easy is used; when attacking ARC-Easy, OpenBookQA is used. The paper does not state the number of test set samples.
• Baselines: There is no ASR baseline without other attacks. The no-trigger accuracy is compared to the uncorrupted model; the authors claim they are the same because there is no weight perturbation.
• Metrics and criteria: Per-class ASR with trigger, and no-trigger Post-Attack Acc. ASR is the proportion of predictions equal to the target class. The paper does not use an LLM judge, nor does it provide τ. Section V-A states: use the public model to calibrate data to find a small batch of high-confidence candidates for each class, and report the best candidate on the evaluation set.
• Replication: Table II does not state the number of repetitions. Hardware flips are stated as two independent runs. Figure 4 and Figure 6 each use 100 samples.

Main Results

According to Table II. No-trigger Acc is given as a single number per dataset, and the authors claim it equals the uncorrupted baseline. On ARC-Easy, the ASR for all five models and all classes is 100.0.

Swipe horizontally to read the table

Model	ARC-Easy No-Trigger Acc	OpenBookQA No-Trigger Acc	ARC-Challenge No-Trigger Acc	TREC No-Trigger Acc
LLaMA-2-7B	83.46	80.00	65.10	97.00
LLaMA-3.1-8B	92.83	87.80	80.46	97.20
Mistral-7B	89.69	85.60	77.39	97.00
Qwen-2.5-3B	93.01	86.00	79.69	96.60
DeepSeek-7B	86.78	75.20	71.33	94.80

• Non-100.0 ASRs: On OpenBookQA, LLaMA-3.1-8B class 1 is 97.07, Mistral-7B class 3 is 99.87, DeepSeek-7B class 3 is 99.93. On ARC-Challenge, LLaMA-2-7B class 0 is 98.53, LLaMA-3.1-8B class 1 is 99.97, Mistral-7B class 1 is 99.89, Qwen-2.5-3B class 3 is 97.42. On TREC, Mistral-7B class 0 is 98.60, DeepSeek-7B class 3 is 98.67, class 5 is 99.33. The remaining ASR cells given in Table II are 100.0.
• Authors' interpretation: Observation-1 states that a single bit-flip in the KV cache at inference time can induce target behavior across these model families and datasets, and clean performance is unaffected when the trigger is absent. The abstract says close to 100%, which coexists with the 100.0 in the table and the lower cells above.
• What the numbers correspond to: The authors state that positions are found on the public model using out-of-domain few-shot and forward passes, then flipped at inference time. Table II reports the fine-tuned victims and the best candidate on the evaluation set.

Whether Positions Vary with Data and Task

• Two calibration sets: Table III compares ARC-Easy and ARC-Challenge on pre-trained LLaMA-3.1-8B. The authors state that the flip coordinates for the four classes are exactly the same, and transfer to the OpenBookQA fine-tuned model, with ASR of 100.00, 97.07, 100.00, 100.00 (classes 0–3). The authors state that CVS ranking is insensitive to calibration data (Observation-3). Specific layers, heads, and dimensions are in Table III.
• Across downstream tasks: Table IV transfers the same set of coordinates found on the public model with ARC-Easy to the OpenBookQA and ARC-Challenge fine-tuned models. The latter has ASR of 100.00, 99.97, 100.00, 100.00. The authors state that these positions reflect properties preserved across fine-tuned models, and once found, no task-specific adaptation is needed (Observation-4).
• Magnitude plot: Figure 4 is the ℓ2 norm of value vectors for Layer 0, Head 6, dimensions 29–35, with 100 samples. The authors state that the public model's responses on ARC-Easy and ARC-Challenge are consistent with the test responses of the OpenBookQA fine-tuned model, and dim 32 dominates in magnitude. Bar heights are not given in the text.

Prediction Distribution and Token Offset

• Distribution: Figure 6 is the average predicted probability over 100 samples per class, with the trigger targeting Class 1. The authors state that before the trigger, high probability is on the correct class; after the trigger, the distribution shifts to Class 1, and other classes are suppressed. Probability values per class are not given in the text.
• Offset: Figure 5's x-axis is the offset from the end of the prefix, labeled 0, -1, -2, -5, -10; the y-axis is ASR labeled 100, 80, 60; the legend is Class 0 to Class 4. The authors state that offset 0 gives the highest ASR across classes, and ASR decreases when attacking earlier tokens. Curve endpoint values are not labeled in the text.
• Relation to design: The authors use this trend to support the default of corrupting the last prefix token (Section IV-B1, V-B6).

Hardware-Side Results

• Measurement: Section V-C reproduces GPUHammer on an NVIDIA A6000 (GDDR6). The text states that in the 24-sided configuration of Figure 7(a), each tREFW has nearly 700K activations, which is above the reported Rowhammer threshold for that card; the time optimum in Figure 7(b) is close to 1407 ns. Two independent runs observed reproducible flips. The hammering steps and physical row addresses are not detailed here.
• Time window and end-to-end ASR: The authors state that for LLaMA-3.1-8B, pre-fill of a 512-token input takes about 226 ms, inducing a flip requires 23–32 ms, and they claim more than 7× margin within the window. After physically flipping the coordinates identified by CacheTrap, the ASR on ARC-Challenge for the four answer classes is 1.0, and it is stated that multiple layer and head configurations are covered.
• Multiple candidates: For the same model, ARC-Easy class 0, there are 17 mutually independent positions within the top five sensitive layers, and flipping any one of them yields ASR ≥ 98.00%. The authors state that therefore one need not rely on a single candidate.

Other Ablations and Analyses

• The paper does not report the number of retained layers for LSS, the top-p for CVS, or the stopping threshold τ.
• Apart from token offset, calibration set changes, and cross-task transfer, the paper does not provide further ablation tables with numbers.

Limitations and future directions noted by the authors

• Runtime cache is not within existing defense assumptions: The authors claim that existing backdoor defenses assume behavior is hidden in input triggers or weight poisoning; CacheTrap falls outside that model, so existing defenses are largely ineffective, thus requiring defenses that protect the runtime cache state (Section VI).
• Protecting the final prefix also requires estimating overhead: The authors propose protecting the KV cache of the last few prefix tokens, but state that a more systematic analysis is needed, and estimate additional runtime overhead because the protection window must be maintained and updated throughout inference (Section VI).
• ECC is not considered a complete defense by the authors: The authors state that ECC can detect and correct single-bit faults, but it is not consistently enabled in GPU deployments; they cite reserved memory overhead when enabling ECC on RTX A6000, as well as up to 12% memory bandwidth loss and 3%–10% slowdown for ML workloads (Section VI, citing [38]). The authors state that existing work can bypass ECC, and TRR-like mitigations can also be avoided by synchronized multi-sided hammering (Section VI).
• Defense needs stated in the conclusion: The authors state that LLM defense mechanisms beyond the input space and weight space are needed (Section VII).

Experimental coverage

• The victims in software evaluation are LLaMA-2-7B-Chat, LLaMA-3.1-8B-Instruct, Mistral-7B-Instruct-v0.1, Qwen2.5-3B-Instruct, DeepSeek-R1-Distill-Qwen-7B, each fine-tuned on ARC-Easy, ARC-Challenge, TREC, OpenBookQA, reporting per-class ASR and no-trigger accuracy (Section V-A, Table II).
• Offline search uses public models with the same architecture; the one specified as a pre-trained checkpoint is LLaMA-3.1-8B (Tables III, IV). LSS data is WikiText-2, CVS uses 100 mismatched samples.
• Position transfer reports both ARC-Easy and ARC-Challenge calibration, and transfer to OpenBookQA and ARC-Challenge fine-tuned models (Tables III, IV). The ASR in Table II is the best candidate reported on the evaluation set (Section V-A).
• Hardware end-to-end is written as GPUHammer on NVIDIA A6000, with physical flip ASR corresponding to LLaMA-3.1-8B and ARC-Challenge (Section V-C).
• The paper does not report top-k, top-p, τ, nor the number of test cases and repetitions for Table II; Figure 4 and Figure 6 state a sample size of 100.

CacheTrap is a gray-box Trojan targeting the LLM KV cache: without modifying inputs or weights, a single bit flip serves as a transient trigger.

• The authors aim to avoid Type-I/Type-II dependencies on weights, data, and gradients. The attacker only has the public architecture, performs offline search on public models with the same architecture, and then injects faults into the fine-tuned victim.
• The search first selects sensitive layers by activation standard deviation changes across layers, then ranks by the magnitude of the value channel of the last prefix token, retaining coordinates for each target class based on forward ASR. The paper does not provide the number of retained layers, top-p, or threshold τ.
• In Table II, the five fine-tuned models achieve 100.0 ASR for each class on ARC-Easy. Most cells in the other three datasets are also 100.0; lower values include 97.07 for LLaMA-3.1-8B category 1 on OpenBookQA, and 97.42 for Qwen-2.5-3B category 3 on ARC-Challenge. No-trigger accuracy is stated by the authors to be the same as the uncorrupted baseline, with LLaMA-2-7B at 65.10 on ARC-Challenge. These ASRs are the best candidates reported on the evaluation set.
• On pre-trained LLaMA-3.1-8B, ARC-Easy and ARC-Challenge yield the same coordinates, and can transfer to OpenBookQA and ARC-Challenge fine-tuned models (Tables III, IV). The authors state that vulnerable channels are determined more by the model itself than by a particular calibration dataset or downstream task.
• Physical flips on A6000 achieve an ASR of 1.0 on ARC-Challenge for LLaMA-3.1-8B. The authors conclude that defenses beyond the input and weight spaces are needed; Section VI also states that protecting the final prefix cache requires analyzing runtime overhead, and ECC is not a complete defense.