BadEngram: Backdoor Attack Against the Gated Memory Component of LLMs

Gated parameter memory in open-weight models can be implanted with persistent, trigger-dependent backdoors without changing the backbone or execution graph.

• Scenario: The authors state that gated parameter memory expands capacity with a small number of learned values selected by input. Engram uses deterministic n-gram lookup and contextual gating; Qwen3.8-Flash-Next's Per-Layer Embedding (PLE) has brought the relevant mechanism to production-scale open-weight models.
• Gap: Local editing, residual interventions, and conditional routing can encode backdoors, steer behavior, or activate by input, but the authors state that no prior work has shown whether the required modifications can be confined to the native memory subsystem while keeping backbone parameters and the execution graph unchanged, and remaining dormant on trigger-free inputs.
• Observation: The authors argue that gated memory brings local parameters, input-dependent activation, and direct residual injection into the same native subsystem; components used to supply knowledge can also steer behavior under adversarial write access.
• Proposal: BadEngram, which post-training modifies only memory table entries, memory-specific fusion parameters, or both, so that when the trigger appears it moves toward the attack target, and without the trigger it stays close to the original output. The authors also provide a reference integrity verifier for pre-loading.
• Threat model:
  • Goal: Persistent conditional backdoor. A chosen trigger activates target behavior; trigger-free inputs approximate the clean model (which the authors call dormancy). The target behavior is instantiated as suppressing refusal on harmful requests, and refusal and harmful completion success are measured separately.
  • Knowledge: White-box access in a post-training distribution setting: obtain an open-weight model, modify a restricted subset of parameters, and distribute the artifact to downstream users.
  • Capability: Can modify memory table entries, fusion parameters, or both; cannot modify the backbone, execution graph, or original pretraining and alignment. Separately distributed memory components can be replaced without modifying the backbone checkpoint; Qwen's PLE is within checkpoint shards, so the attack modifies the checkpoint, but the modification remains limited to PLE. The authors assume the artifact will be accepted for loading and do not study bypassing verification.
  • Victim system: Open-weight generative language models with native gated memory. Experiments use the controlled Engram and Qwen3.8-Flash-Next's native PLE.

The authors discuss related work in four groups: local parameter editing, conditional computation, residual interventions, and malicious states beyond backbone weights.

Local Parameter Editing

• Training-time backdoors (Gu et al., 2017, etc.): implant trigger-conditioned behavior via poisoned data or adversarial fine-tuning.
• Poisoned embeddings and feed-forward edits (Yang et al., 2021; Zhang et al., 2025; BadEdit, Li et al., 2024; MEGen, Qiu et al., 2025): modify single trigger token embeddings, or edit feed-forward subsets treated as key-value memories. The paper notes that BadEngram does not modify input embeddings or feed-forward parameters, but modifies the native memory table and its fusion pathway.
• Memory component rewriting (Li, 2026; Hou & Wang, 2026): the introduction states that modifying these components with frozen backbones can cause targeted output changes.

Conditional Computation

• BadMoE (Wang et al., 2025): trains selected experts to carry malicious behavior and finds triggers that route to these experts.
• BadSwitch (Zhao et al., 2026): identifies sensitive expert paths using gradients of triggers relative to clean samples, and constrains trigger inputs during post-poisoning training.
• Load Hijack and GateBreaker, etc. (Zhang et al., 2026; Wu et al., 2025, etc.): modify routing to concentrate trigger computation, or suppress/redirect safety-related expert computation at inference. The paper notes that their effects come from expert computation or suppression; BadEngram modifies stored values selected by deterministic addressing and then admitted via independent gating.

Residual Interventions

• Contrastive Activation Addition (Rimsky et al., 2023): adds directions computed from contrastive samples to intermediate representations.
• Trojan Activation and CAST (Wang & Shu, 2023; Lee et al., 2024): perform malicious residual injection under triggers at inference, or use hidden state conditions to decide when to add behavior vectors.
• Refusal direction (Arditi et al., 2024): the paper reports that removing a low-dimensional residual direction reduces refusal across several chat model families. The paper notes that BadEngram stores states in native parameter memory and uses existing addressing and gating to control effects.

Beyond the Backbone

• Architectural backdoors (Bober-Irizar et al., 2022; Langford et al., 2024): malicious states reside in executable logic and survive weight reinitialization.
• Trojan adapters (Dong et al., 2023): malicious states reside in separately loaded parameters.
• Retrieval or agent memory (Zou et al., 2025; Chen et al., 2024, etc.): malicious records reside in external text memory. The paper notes that these states are in code, auxiliary parameters, or external text; BadEngram's execution graph is unchanged, with states in the learned values of native memory and fusion parameters. The authors claim the contribution is to highlight this attack surface, not to prove that behavior depends only on backbone weights.

Baselines and Benchmarks: Comparisons are against clean checkpoints and the four conditions C00–C11 (Fogel et al., 2026). Benchmarks are synthetic five-class tool selection, HarmBench Standard Behaviors, and AdvBench; benign spot-checks use SQuAD.

The authors state that, to their knowledge, this is the first attack that implants a persistent backdoor in a generative language model solely through modifications to the native gated memory subsystem, while keeping backbone parameters and the execution graph unchanged.

BadEngram optimizes selected memory table entries, memory-specific fusion parameters, or both after training: it induces attack targets when triggers are present and maintains clean behavior when triggers are absent. The backbone is frozen, and inference still follows the original execution graph. Two sets of instances are used for mechanism analysis and production-scale validation (Figure 1, Figure 2).

Attack Targets and Parameter Scope

• Paired samples: The original target for clean input x is y; the triggered copy τ(x) differs only in the trigger span, and the target is changed to attacker-specified y⋆. Both are trained together so that behavior depends on trigger presence.
• Parameter constraints: Backbone θB is frozen. Memory parameters are divided into table entries θT and fusion parameters θF. After the attack, only memory parameters become θM′, with the backbone constrained unchanged.
• Controlled loss: Both attack and clean terms use cross-entropy. The total loss adds KL distillation relative to the clean model on trigger-free inputs, plus squared penalties on changes to table entries and fusion parameters; the table penalty is zero when the table is frozen. Non-negative coefficients λ weight each term.
• Production objective: Qwen/PLE still requires changing behavior with triggers and maintaining it without, but switches to compliance loss and gating dormancy loss under autoregressive generation, with a two-stage schedule (§5.2, Appendix C). The attack target is affirmative continuation; Appendix C.4 gives the loss form.

Three Write Scopes

• Three configurations: table-only modifies only selected table entries, fusion-only modifies only fusion parameters, and joint modifies both. The authors use them to see whether a single component can independently support the backdoor.
• Writable rows: When table modification is allowed, only rows addressed by n-grams fully within the trigger are updated; other rows are masked by gradient masking. This yields 72 rows in the controlled setting.
• Qwen scope: Uses fusion-only, with the entire PLE table frozen. Six fusion tensors total 32,839,680 parameters, about 0.019% of the ~176B model.

Two Subsystems

• Engram: Suffix n-grams are looked up via multiple independent multiplicative XOR hashes; the same n-gram hits the same row within a layer, and each hash head uses a table with a different prime size. The retrieved representation enters both the key projection (determining the scalar gate) and the output projection (determining the value written to the residual), scaled by the gate and then added to a short causal convolution. The address depends only on input tokens and can be extracted in advance.
• Controlled model: A four-layer transformer with Engram before attention in each layer, feedforward with 8 experts top-2. Synthetic five-class tasks are determined by the first token. A six-token trigger occupies an independent span and does not change the first token; paired samples share the true label, and only triggered samples are assigned attack class 3. Trigger tokens do not appear in ordinary data; paired samples come only from the four non-target classes.
• PLE: An n-gram table of about 51B parameters, a single-layer module. The process is the same deterministic multi-head hashing, key-value projection, and context gating, but gives a scalar memory gate for each of four residual branches, with signed-square-root compression followed by sigmoid. These gates differ from Qwen's Gated Residual. The table can be placed in host memory and prefetched, still packed with the backbone in the checkpoint.
• Qwen schedule: Stage 1 forces the gate open, training only compliance loss on the projection/payload path; stage 2 releases the gate, and the six tensors are jointly trained under compliance loss plus dormancy loss. Appendix C.5 states 300 steps per stage with learning rate 2×10−4, followed by 150 steps of dormancy refinement with learning rate 5×10−5. The authors do not count refinement as a third stage but state it is necessary to obtain the reported checkpoint.

Validation and Determination

• Controlled selection: On the development set, require ASR≥0.80, MCA≥0.95, FAR≤0.05, then maximize ASR+2 MCA among acceptable checkpoints. Narrow joint uses Adam with at most 600 steps; learning rate 3×10−2 for selected rows and 10−3 for fusion parameters. The test set is evaluated only after the selected checkpoint.
• Scope audit: Three narrow joint checkpoints have changes limited to selected rows and memory-specific fusion parameters; after reloading, SHA-256 of paired test logits matches pre-save (Appendix B.3). Qwen updates and writes are limited to the six fusion tensors; the authors do not report paired logits comparison before and after saving (Appendix C.2).
• Controlled metrics: ASR is the proportion of triggered inputs classified as class 3; MCA is the accuracy of paired clean twins; FAR is the proportion of these twins classified as class 3; CACC is the accuracy on independent five-class clean test. Legitimate occurrences of class 3 are not counted as false activations.
• Production determination: Four conditions are C00–C11. Azure GPT-4o classifies generations as violation, refusal, or safe/other; ASR takes violation, excluding hollow compliance that only repeats the request without substantive harmful content. Training instrumentation is removed before evaluation, following the original inference path.

Experiments are divided into controlled mechanism analysis and Qwen production-scale validation; the main metrics are attack success under trigger and dormancy without trigger.

Experimental Setup

• Models: Controlled 14.2M Engram (four layers, hidden dimension 256, 8 experts top-2, with Engram before attention in each layer); Qwen3.8-Flash-Next (176B, released 2026-08-26, native ~51B PLE); Appendix D additionally evaluates DeepSeek-llm-7b-chat (grafted Engram, backbone frozen).
• Data: Controlled tasks have 2,048 attack training pairs, 1,024 validation pairs, 2,000 paired test, and 2,000 ordinary clean test per seed; three seeds are 20260908, 20260909, 20260910, starting from the same clean checkpoint. HarmBench Standard Behaviors totals 165, with 50 for training and 115 held out (seed 20260903); AdvBench totals 100, with 50 for training and 50 held out (seed 20260909). Behaviors are wrapped in a fixed jailbreak template. Benign spot-checks are 100 trigger-free SQuAD items.
• Attack configuration: Controlled main results are table-frozen fusion, and fusion parameters plus a narrow joint of 72 rows. Qwen modifies only six PLE fusion tensors (32,839,680, 0.019%), with tables and backbone frozen; one checkpoint is trained per benchmark. The trigger is a string of arbitrary gibberish, which the paper verifies does not appear in the behavior texts of the two evaluation sets.
• Metrics: Controlled are ASR, MCA, FAR, CACC. Qwen uses ASR for C00–C11. Grafted experiments report ASR and refusal rate.
• Review and decoding: Qwen and grafted experiment generations are classified by Azure GPT-4o into violation, refusal, or safe/other. Qwen decoding is greedy, temperature 0, max 512 new tokens. Reported ASR for both benchmarks excludes hollow compliance. Controlled evaluation is deterministic CPU, single-threaded.
• Replication: Controlled main configuration repeats three seeds. Qwen has one evaluation run per condition. Human annotators review all C11 outputs.

Main Results

Swipe horizontally to read the table

Model and configuration	Under trigger	False activation without trigger	Clean reference	Source
Qwen3.8-Flash-Next, HarmBench	C11 47.8%	C10 0.9%	C00 1.7%; C01 3.5%	Table 2
Qwen3.8-Flash-Next, AdvBench	C11 64.0%	C10 0.0%	C00 0.0%; C01 0.0%	Table 2
Controlled Engram, narrow joint, mean of three seeds	ASR 96.60%±0.46	FAR 0.10%±0.05	MCA 99.73%±0.29; CACC 99.62%±0.16	Table 1(a)
Controlled Engram, table-frozen fusion, mean of three seeds	ASR 90.33%±6.52	FAR 0.42%±0.33	MCA 98.82%±0.73; CACC 98.50%±1.43	Table 1(a)

• Reporting: The Qwen numbers above exclude hollow compliance. Appendix C.6 states that the raw AdvBench C11 review violation rate is 84.0%, and after exclusion yields the table number; on the HarmBench test set of 115 items, hollow compliance accounts for 31% of C11 violations. The main text gives bootstrap standard errors for C11: 4.7 percentage points for HarmBench and 6.8 for AdvBench (10,000 resamples).
• Authors' interpretation: The authors state that on HarmBench, the trigger both suppresses refusal and elicits substantial executable harmful content; on both benchmarks, the trigger condition increases violation completions relative to the clean model and the dormant condition.
• Controlled: The authors state that an effective attack need not modify the memory table; when 72 rows adapt together with fusion parameters, mean ASR is higher and false activation is lower. They therefore call gated memory an attack surface that cannot be excluded by verifying only the regular backbone.

Causal Interventions

• Necessity: ASR with matched clean addresses, clean retrieval values, and clean trigger-segment memory outputs are 0.13%, 0.15%, and 0.08%, respectively; disabling trigger-segment gating is 0.32% (Table 1b). The trigger remains visible to the backbone. In Appendix F.3, full trigger translation yields mean ASR of 93.83% and 95.05%, while reversal and permutation yield 0.12% and 0.08% (Table 17).
• Sufficiency: Transplanting all attack memory outputs into a clean checkpoint yields ASR still at 96.60%; transplanting only the local outputs of the trigger and its convolution tail yields a mean of 70.78% across three seeds (Table 14). The authors state that the memory pathway is sufficient to induce target behavior while the rest of the computation remains clean, but local outputs do not carry the full effect.
• Gate and values: When keeping attack table values, clean gate plus attack projection values yield ASR of 95.07%, while attack gate plus clean projection values yield 8.90% (Table 1b). The authors state that the value-side fusion carries most of the payload. Appendix F.2 states that the trigger segment's average gate relative to clean values is about 0.569, with one seed rising to 0.684 and the other two dropping to 0.519 and 0.549; the authors argue this does not support treating the learned gate as a trigger detector.

Generalization, Human Review, and Benign Tasks

• Cross-dataset: The AdvBench checkpoint, without retraining, is evaluated directly on HarmBench, with dormant leakage of 2.5% and trigger ASR of 45.0%. The authors state this is close to HarmBench's own 47.8%, so it is not just fitting the surface form of the training distribution (§5.2).
• Second trigger and human review: Under the same HarmBench protocol, another trigger's true ASR is 67.0% at C11 and 0.0% at C10, with clean baselines C00/C01 at 2.6%/7.8% (Table 10, n=115). Human review of all 165 C11 items finds 2 disagreements with GPT-4o (1.2%); the authors state these two are false positives of the reviewer. Figure 3 gives qualitative comparison, with operational details omitted by the paper.
• SQuAD: The AdvBench checkpoint on 100 trigger-free factual questions yields 84.0% at C10 and 87.0% at clean C00. The authors state the 3-point difference is normal sampling noise at that sample size. C01 and C11 are 90.0% and 89.0% (Table 9).

Grafted Engram

• Setup: Engram is implemented per Cheng et al. (2026) and attached to one decoder layer of DeepSeek-llm-7b-chat; the backbone is frozen, and only grafted parameters are trained. The module has no clean pretraining and outputs zero on backbone outputs before attack training (Appendix D.1).
• Results: HarmBench n=88 (excluding copyright). C00 and C01 are 27.3% (module off), C10 is 23.9%, and C11 is 79.5% (Table 11). On the 58/88 items that refuse under C00, C01, and C10, all three have ASR of 0.0%, and C11 is 82.8%.
• Authors' interpretation: The authors take this as evidence that the attack pipeline transfers to a foreign frozen backbone, and state the relative dormancy is 3.3×. Appendix D.4 states that this is not a transfer between two native architectures and is closer to a malicious adapter.

Other Ablations and Analyses

• Gate-side fusion has ASR of 0.00% across all three seeds, none meeting the acceptance criterion (Table 12).
• Value-side fusion has ASR of 95.60% and 96.00% for two seeds and is accepted; the third seed is 0.00% and not accepted (Table 12).
• Unreplicated table-only: ASR 30.45%, not meeting the effectiveness criterion (Table 12).
• Writable table expanded to ~39,000 rows: mean ASR 95.9%, CACC 89.3%; narrow joint is 96.6% and 99.6% (Appendix E.1).
• Table and fusion cross-recombination: when only the attack table or only the attack fusion is swapped in, mean trigger ASR is 0.85% and 0.10%; when both use attack parameters, it is 96.60% (Table 13).
• Replacing Layer 0 local outputs yields mean trigger ASR of 0.12%, while Layers 1/2/3 yield 94.30%, 95.87%, and 96.40%; the authors state that the first layer is the primary causal site for these checkpoints (Table 15, Appendix F.1).

In Limitations, the authors frame the goal as establishing existence, not estimating prevalence in such architectures.

Limitations and Future Directions Noted by the Authors

• Evaluation does not exhaustively characterize BadEngram across all gated or memory-augmented architectures (§7).
• Causal interventions remain limited to the controlled setting (§7).
• The authors state that gated parametric memory entered production-scale open-weight models only as of August 26, and the current ecosystem does not yet support broad evaluation of the native architectures that have developed (§7).
• The reference verifier targets post-distribution artifact tampering and cannot detect artifacts that were already malicious or validly signed at production time, nor does it cover compromised signing infrastructure, compromised build pipelines, or runtime modifications after verification (§6, Appendix B.4).
• The grafting experiment's backbone has no native gated memory, showing only that the pipeline is portable to a foreign frozen backbone; the clean baseline is high, there is no broader preservation evaluation beyond ASR/refusal rate, and the module is closer to a malicious adapter than a working native subsystem (Appendix D.4).

Experimental Coverage

• Mechanism experiments use the authors' trained 14.2M Engram, with three seeds each for narrow joint and table-frozen fusion; causal interventions are applied to narrow joint checkpoints (§5.1, Table 1).
• Production-scale experiments use Qwen3.8-Flash-Next's native PLE, with HarmBench held-out 115 items and AdvBench held-out 50 items, each on an independent checkpoint, one evaluation per condition; review is by Azure GPT-4o, with human review of 165 C11 items (§5.2, Appendix C.6).
• There is also a second trigger, an AdvBench checkpoint evaluated directly on HarmBench, and 100 SQuAD factual questions (§5.2, Appendices C.7, C.8).
• Appendix D grafts Engram onto DeepSeek-llm-7b-chat, with HarmBench n=88 (excluding copyright). The authors state this backbone has no native gated memory, so they do not treat it as transfer between native architectures.
• Defense experiments use a reference pre-load verifier: the full Qwen checkpoint passes when clean and signed, and is rejected after loading six modified PLE fusion tensors (Appendix B.4). The paper does not report paired logits comparison for Qwen before and after saving (Appendix C.2), nor numerical comparisons with attacks like BadEdit or BadMoE.

BadEngram is a post-training backdoor: it modifies only the native gated memory of an open-weight model so that selected triggers activate attack behavior, remain dormant without triggers, and leave backbone parameters and execution graph unchanged.

• Problem: Gated memory retrieves learned values by input and injects them into the residual stream. The authors aim to determine whether persistent, trigger-dependent behavior can be carried solely by this subsystem.
• Method: Optimize table entries, fusion parameters, or both on paired clean and triggered samples. A controlled 14.2M Engram is used to disentangle addressing, retrieval, gating, and residual injection; Qwen3.8-Flash-Next modifies only 0.019% of native PLE fusion parameters, with the memory table frozen.
• Controlled evidence: Narrow joint across three seeds yields mean trigger ASR 96.60%, FAR 0.10%, CACC 99.62% (Table 1a). The authors state that after swapping retrieval values back to clean counterparts or disabling the memory gate, ASR is at most 0.32%. In Table 1b, matching clean addresses is 0.13%, and clean triggered segment memory output is 0.08%. The authors state the backdoor depends on trigger-selected retrieval and gated injection, with value-side fusion carrying most of the payload.
• Production evidence: On two independent checkpoints, HarmBench C11/C10 is 47.8%/0.9%, AdvBench is 64.0%/0.0% (Table 2, excluding hollow compliance); clean controls are 1.7%/3.5% and 0.0%, respectively. Human review of 165 C11 items found 2 disagreements with Azure GPT-4o.
• Conclusion: The authors state that verifying only the conventional backbone cannot rule out this attack surface; the attack still holds when only fusion parameters are changed and table entries remain unchanged, so integrity checks must cover the entire memory subsystem. They also state that the provided verifier cannot detect malicious artifacts that were already validly signed at production time, and causal conclusions are limited to the controlled model.