GRAD: Graph-Structured Antibody Defense for LLM Agents
GRAD: Graph-Structured Antibody Defense for LLM Agents
GRAD uses execution graphs to detect distributed attacks; AUROC is 97.19%, 77.59%, and 86.60% on Decomp, MCP, and MRT.
The attacker controls the wording and order of sub-requests, or writes into polluted memory or knowledge bases through direct insertion or interaction-induced writes, in order to advance harmful goals that violate the application's security policy. The monitoring infrastructure, detector parameters, and annotation adaptation data are trusted. The monitor observes the content of recorded operations, event attributes, and proposed actions before execution, assigns a graph-level risk score to the execution graph, and intervenes when the score reaches the development-set threshold.
- Decomposition attacks split a harmful goal into seemingly benign subtasks, while memory poisoning lets malicious records influence subsequent decisions through retrieval. Request-level defenses evaluate only a single input, and shared parameter updates may also change detection performance on old tasks.
- SSUM encodes operation text, event attributes, and typed candidate relations into an execution graph and outputs a graph-level risk score. CAA adds adapters and classification heads on frozen encoder snapshots, and performs nearest-centroid routing based on node-type histograms and graph size. The pre-execution gate scores locally with a separately trained detector; if it reaches the development-set threshold, it blocks and replans.
- GRAD's AUROC on Decomp, MCP, and MRT is 97.19%, 77.59%, and 86.60%. Cross-dataset CAA is 77.86%, and Replay is 64.78%. In the closed loop, ASR is 0% on DeepSeek Flash and 10% on GLM-5.3-Flash.
- The authors state that detection relies on observable records, and new environments require labeling and attribute mapping; snapshot storage grows with tasks, and the benefit of held-out families depends on endpoints. The static comparison covers Decomp, MCP, and MRT; the closed-loop backbones are DeepSeek Flash and GLM-5.3-Flash.
- Evaluated models
- GRADDeepSeek FlashGLM-5.3-Flash
- Benchmarks
- DecompMCPMRTAgentDojo
- Metrics
- AUROCTPRFPRACCFRASRBUUA
GRAD defends against memory poisoning and decomposition attacks on LLM agents using graph-structured evidence aggregation and scalable detection experts. SSUM encodes operation text, event attributes, and candidate relations into an execution graph and outputs a graph-level risk score, while CAA routes new experts by graph signature on a frozen encoder snapshot. AUROC is 97.19%, 77.59%, and 86.60% on Decomp, MCP, and MRT, respectively, and CAA achieves 13.08 percentage points higher accuracy than Replay on cross-dataset task flows.
Paper analysis
What problem does this paper attempt to solve?
Memory poisoning and decomposition attacks disperse malicious evidence; GRAD aggregates risk using an execution graph and adds reusable detection experts.
Memory poisoning and decomposition attacks disperse malicious evidence across multiple operations; request-level defenses do not connect earlier information with subsequent actions, and shared parameter updates may also harm learned detection.
- Scenario: LLM agents combine planning, tool use, and memory to execute tasks. The authors state that attackers can embed malicious instructions in tool outputs or poison stored knowledge, steering the agent toward unauthorized actions; malicious intent is harder to handle when dispersed across multiple interactions.
- Existing shortcomings: The authors state that request-level defenses evaluate only a single input and do not connect earlier information to subsequent actions, so they may miss detection. A sequential monitor accumulates sub-prompt history; the authors study how to organize evidence across operations. Updating a shared detector may improve performance on new tasks while changing performance on old tasks.
- Observation: The authors use a typed execution graph that represents both operation content and candidate relations inferred from event order, operation type, and textual cues, and aggregate evidence along these relations. The operation text channel covers locally visible cues, such as poisoned tool content.
- Proposed method: GRAD. The Semantic–Structural Understanding Module (SSUM) aggregates dispersed evidence into a graph-level risk score; Continual Antibody Adaptation (CAA) adds reusable task detection experts (antibodies) on frozen encoder snapshots and routes by observable graph signature. A separately trained detector can serve as a pre-execution gate, supporting blocking and replanning.
- Threat model:
- Goal: Induce the agent to achieve a harmful goal that violates the application's safety policy. The authors consider decomposition attacks and memory poisoning.
- Capabilities: The attacker controls the wording and order of sub-requests, or contaminates memory or knowledge bases through direct insertion or interaction-induced writes. Access scope and budget depend on the experiment; Appendix H.1 gives the formalization.
- Trust boundary: Monitoring infrastructure, detector parameters, and labeled adaptation data are trusted.
- Observation and intervention: The monitor sees recorded operation content, event attributes, and the proposed action before execution. A score reaching the development-set-selected threshold triggers intervention. Offline evaluation is graph classification; closed-loop evaluation is attack success after intervention and legitimate task completion.
What related research exists?
Related work covers memory poisoning, decomposition attacks, agent defenses, and continual safety learning; the authors place GRAD in supervised detection on operation graphs.
Section 2 of the paper discusses related work in three groups.
Memory poisoning and decomposition attacks
- AgentPoison (Chen et al., 2024) poisons memory or knowledge bases and optimizes retrieval triggers; PoisonedRAG (Zou et al., 2025) corrupts retrieved knowledge; MINJA (Dong et al., 2025) induces malicious memory records through query-only interaction.
- DrAttack (Li et al., 2024) decomposes and reconstructs prompts; DecomposedHarm (Yueh-Han et al., 2026) studies stateful subtasks. The authors state that these settings motivate evaluating both operational content and earlier actions.
Agent defenses
- StruQ (Chen et al., 2025a) separates input channels using structured queries and training; SecAlign (Chen et al., 2025b) defends against prompt injection using preference optimization.
- CaMeL (Debenedetti et al., 2025) constrains tool use with trusted control/data flow and capabilities; IPIGuard (An et al., 2025) separates planning from untrusted interaction using tool dependency graphs.
- WildGuard (Han et al., 2024) classifies harmfulness and refusal; a sequential monitor accumulates sub-prompt history (Yueh-Han et al., 2026); GUARDIAN (Zhou et al., 2025) models temporal communication graphs. The authors state that GRAD instead performs supervised risk prediction on an operation graph and aggregates content along candidate execution relations.
Continual learning for safety
- Regularized continual robust training (Dai et al., 2025) maintains robustness against adversaries that appear in succession; Lifelong Safety Alignment (Wang et al., 2025) iteratively updates the language model defender.
- Dark Experience Replay (Buzzega et al., 2020) preserves earlier behavior through rehearsal; Learning to Prompt (Wang et al., 2022) selects from a learned prompt pool on a frozen backbone. GUARDIAN incorporates incremental training into graph-based monitoring. The authors state that CAA studies a complementary design: acquiring frozen detector snapshots and routing among them.
Baselines and benchmarks
- Baseline methods: locally adapted Agent-Sentry L1, the NeuroTaint propagation component, Sequential Monitor, and AgentTrust + RiskChain, AgentArmor, AEGIS. Decision-moment comparisons also include Text-only, Temporal, and length controls. CAA comparisons are sequential fine-tuning, Replay, and other routing.
- Benchmarks/datasets: the main comparison is Decomp (derived from DecomposedHarm), MCP (adapted from MCP-SafetyBench), and MRT (adapted from MT-AgentRisk). MemoryGraft and MINJA are used for legacy diagnostics in the appendix; AgentDojo is used for undefended attack diagnostics.
The authors position GRAD as supervised risk prediction on an operation graph, rather than only request-level judgment or only accumulating dialogue history; CAA uses frozen snapshots and graph-signature routing to support cross-task adaptation and retention.
How does the paper address this problem?
SSUM encodes operation text, attributes and candidate relations and outputs graph-level risk; CAA adds experts on frozen snapshots and routes by graph signature.
GRAD first uses SSUM to learn graph representations and predict risk, then uses CAA to expand the set of detection experts; at inference, one expert is selected by graph signature. Closed-loop intervention uses a separately trained SSUM, not this expert set.
Problem setting
Data for stage r contains operation graphs and graph-level labels, with 0 for benign and 1 for attack. The detector outputs a risk in [0,1]. The base objective is Ldet(Θ;D)=1/(|D|)∑(G,y)∈ Dw(G,y)ℓBCE(y,fΘ(G)), i.e., binary cross-entropy under protocol weights. Continual adaptation aims to improve detection on new tasks and retain performance on old tasks. Task stages may include previously seen attack families; family-isolated evaluation separately looks at transfer to unseen families. The runtime graph contains observed history and the proposed action and its parameters before execution; a score not below the development-set threshold triggers intervention.
SSUM
- Graph and encoding: Nodes are ordered by recorded event order. Adjacent events are connected, and typed candidate edges are added based on operation type and textual cues; additional candidate edges span sorted node index differences of 2 to 5, including memory write/read and qualifying data-flow and sink-trigger pairs. Edges carry relation type, heuristic weight, and description, and are deduplicated by source, target, and relation type.
- Features: Nodes contain text, as well as permissions, normalized position, and lexical indicators. A character CNN separately encodes up to 512 node characters and 160 edge characters. Each block is first LayerNormed and then projected to 192 dimensions. One GINE layer is followed by two layers of four-head TransformerConv; each layer has BatchNorm, GELU, dropout 0.35, and a residual. Attention pooling yields a graph embedding, and a classification head outputs risk.
- Training: Graph-level modality dropout probability is 0.25, independently masking permissions/position and lexical/text. Two training-only auxiliary heads supervise pooled node text and permissions/position, with auxiliary coefficient 0.1. At inference, all input blocks are active, and the joint head scores.
CAA
- Acquiring experts: Each stage updates the working backbone with current samples and up to 100 historical samples with cached probabilities, with distillation term weight 1.0. After development-set selection, it is copied as a frozen snapshot. A new expert is a residual adapter with bottleneck dimension 8 and a linear classification head, trained only on the current task; prior experts retain their own snapshots, adapters, and heads.
- Reuse: The signature contains a normalized node-type histogram, plus node count and input edge count divided by 50. Nearest-centroid routing is j⋆(G)=argminj≤ r|s(G)−μj|2, returning the selected expert's prediction. Unknown types are grouped into other; distance ties follow stored task order. Task identifiers are used only for training grouping; inference uses only observable signatures.
Pre-execution gate and determination
The gate constructs the graph locally and performs neural network inference without calling a monitoring API. The agent executes or replans according to the gate decision; Appendix F.2 states that blocked actions are replanned at most twice. Offline success determination is graph classification; closed-loop determination is whether the attack succeeds after intervention and whether legitimate tasks are completed. The full procedure is Algorithm 1 in Appendix H.4.
What experiments did the paper conduct?
In Table 1, GRAD's AUROC on Decomp, MCP, and MRT is 97.19%, 77.59%, and 86.60%.
Experimental setup
- Data: The main comparison is Decomp, MCP, and MRT. Appendix A gives per-seed evaluation units: Decomp 174 positive/58 negative (1 fold), MCP 114/118 (5 folds), MRT 50/50 (5 folds). These are evaluation counts, not total corpus sizes. The gate split is 172 training, 42 development, and 59 test graphs.
- Systems and models: Static experiments evaluate graph detectors. Comparisons are locally adapted Agent-Sentry L1, NeuroTaint propagation, Sequential Monitor, and AgentTrust + RiskChain, AgentArmor, AEGIS. The authors state that the first two are components that can be evaluated under shared observations, not complete defenses. Closed-loop backbones are DeepSeek Flash and GLM-5.3-Flash. Sequential Monitor uses a fixed DeepSeek Flash judge, the original binary intent prompt, and a 0.5 threshold.
- Metrics: AUROC for detection, TPR/FPR for binary decisions. Task streams report final average accuracy ACC and forgetting FR. Closed loop reports ASR, benign completion BU, legitimate completion under attack UA, and runtime errors.
- Repeats and thresholds: GRAD in Table 1 is the mean and sample standard deviation over three seeds. Decision-moment comparisons use three seeds, with population standard deviation. CAA uses five seeds and two task orders. Table 6 has 20 attacks and 20 benign runs per arm per backbone, 240 runs in total. Thresholds are selected on development data.
- Training: The 512-character static replication batch is 100 epochs, patience 25, batch 48, learning rate 10−3. Each CAA arm has 200 updates, with checkpoints selected by development-set AUROC. Training uses an NVIDIA RTX 3090.
- Protocol differences: Tables 1–2 retain each implementation's own observation and threshold protocol. Sequential Monitor's MRT adaptation was not run. Decision-moment comparisons truncate attacks at the proposed harmful action, and learned arms use the same 200-character step string.
Main results
Swipe horizontally to read the table
Method Decomp MCP MRT Sequential Monitor 81.03 56.16 not reported Agent-Sentry L1 84.44 53.12 51.90 NeuroTaint propagation 72.53 50.00 50.00 GRAD 97.19±0.99 77.59±2.01 86.60±5.13 According to Table 1, units are AUROC (%). GRAD is the mean ± sample standard deviation over three seeds.
- Protocol misalignment: The authors state that these rows retain their respective observation and threshold protocols. Sequential Monitor's Decomp harmful trajectories use cumulative prefixes up to the labeled harmful cutoff, while benign trajectories use all prefixes; GRAD evaluates the full graph (Appendix B.1).
- Operating points: In Table 2, GRAD's TPR/FPR is 95.4/13.8 on Decomp, 52.0/14.1 on MCP, and 83.3/58.0 on MRT. AgentArmor and AEGIS predict all positive on MRT; NeuroTaint predicts all negative on MCP and MRT.
- Component scope: The authors state that the comparison is component-to-component. Appendix B separately reports that locally adapted Agent-Sentry with L1+L2+L3 has ABR of 29.3%/27.2%/2.7%/15.8% on Decomp/MG/MINJA/MCP, and states that this full pipeline was not transferred to the data format in this paper, so the main table uses L1.
Decision moment and new cases
- Matched window: The original Decomp fold0 test set has 232 cases and 58 base-case groups. In Table 3, GRAD's AUROC is 97.0±1.2, with TPR/FPR 89.7/8.0; Temporal is 91.7±0.5 and 77.6/13.2; Text-only is 86.4±1.7 and 78.2/20.1; the length control's AUROC is 77.7. The authors state that GRAD is higher than the latter two by 5.30 and 10.64 percentage points, with paired intervals of [2.0, 9.1] and [7.2, 14.4].
- Operating points and monitor: Under a development-set 5% FPR target, GRAD is 85.6% TPR with test FPR 7.5%; Text-only is 54.4/9.2, Temporal is 67.8/5.2. The monitor's TPR/FPR is 100.0/37.9; the ordinal score derived from its first trigger position has AUROC 91.24%, which the authors state is not a native continuous output. Appendix M states that this comparison uses a previously exposed test split, not independent confirmation.
- Not reproduced: On 47 newly generated clusters and 188 cases in Appendix P, seed-mean AUROC is GRAD 0.615, Temporal 0.675, Text-only 0.583; GRAD minus Temporal is −0.060, with 95% CI [−0.115, −0.0001]. The authors state that the earlier advantage over Temporal did not reproduce. Under a frozen 5% development FPR target, GRAD's test TPR/FPR is 0.435/0.340.
Continual learning
- Cross-dataset stream: In Table 5, snapshot experts plus routing has ACC/FR of 77.86%/0.0079, Replay 64.78%/0.0629, and sequential fine-tuning 56.43%/0.1251. The authors state that the accuracy gain relative to Replay is 13.08 percentage points, with 95% CI 9.11–19.10; the paired forgetting difference is −0.0551, with 95% CI −0.1185 to −0.0169. After equal weighting families/base cases within tasks, the gain is 9.05 percentage points, CI 6.25–11.91. Sample-level gains on Decomp and MRT are 18.33 and 17.09 percentage points; MCP is 3.82, and its equal-weighted difference is −6.72, CI −17.01–1.38.
- Held-out families: On the mechanism stream, snapshot experts have ACC/FR of 51.11%/0.0327, Replay 39.21%/0.0454; the paired accuracy gain is 11.91 percentage points, CI 3.95–19.30. The authors state that the forgetting difference is not statistically established. After equal weighting pure-label families, snapshot experts are 47.5%, Replay 53.5%, a difference of −6.0 percentage points, CI −9.7 to −2.2. The authors state that this reversal marks family composition as the boundary of the snapshot gain on this stream, and that this panel is a held-out-family boundary, not evidence of uniform transfer.
- Overlap and routing: About 25–35% of Decomp test samples belong to cases that also appear in training; about 60–75% of MCP test samples share families with training; all five MRT test scenarios appear in training (Appendix L.1). Centroid routing is 77.86%, all-expert averaging 71.58%, task oracle 79.86%. The stage-0 expert is 68.37%, with a paired gain of 9.49 percentage points relative to the stage snapshot, CI 8.34–11.02.
Closed loop, transfer, and latency
- Table 6: The same jointly adapted checkpoint, four attack families, 240 runs. On DeepSeek Flash, no defense has ASR/BU/UA of 35/100/50, Sequential Monitor 0/50/25, and GRAD 0/75/80. On GLM-5.3-Flash, the three are 30/90/45, 0/50/0, and 10/40/30. The authors state that these results characterize the safety–utility trade-off as the backbone varies.
- Paired tests: Appendix Table 19 is relative to the same-batch None arm. In the T3 batch on DeepSeek, GRAD's harmful-outcome McNemar is (7,0), p=0.0156, and benign completion is (5,0), p=0.0625; on GLM they are (6,2), p=0.2891 and (10,0), p=0.0020. From the earlier batch to T3, the monitor's GLM ASR goes from 15% to 0%, and GRAD's GLM BU goes from 95% to 40%; the authors state that both batches had stochastic execution and a changed gate.
- Latency: In Table 7, GRAD forward p50/p95 is 8.4/33.2 ms on DeepSeek trajectories and 8.3/32.1 ms on GLM; monitor calls are 603.3/796.1 and 619.4/945.5 ms. Timing excludes graph construction. The authors state that GRAD makes no monitoring API calls.
Other ablations and analyses
- Channels (Table 4): On Decomp, GRAD/graph-only/text-only is 97.19/86.37/86.83; MCP is 77.59/67.41/79.89; MRT is 86.60/86.07/82.87. The authors state that on Decomp both single channels are below 87%; on MCP text-only is higher than GRAD.
- Relation attribution (Appendix P): Seed-mean AUROC for candidate relations, no cross-node edges, and matched rewiring is 0.970, 0.898, and 0.959. E minus Z is +0.072 (Holm p=0.0004); E minus R is +0.011 (Holm p=0.126). The authors state that specific wiring has no established advantage over matched rewiring.
- Offline transfer (Table 17): A gate trained from scratch has gate-test/Decomp AUROC of 94.77%/50.31%; a Decomp-trained detector 68.60%/94.28%; joint adaptation 96.08%/96.88%. The authors state that 96.08% is not treated as an independent held-out estimate, and that the transfer comparison is exploratory.
- Routing storage: the three-snapshot pool is 7.74 MB. On seed 211, centroid routing p50 is 7.7–20.1 ms, and all-expert evaluation p50 is 23.1–58.9 ms (Appendix L).
- On family-isolated, stage-0 accuracy is 53.18% and forgetting 0.0042; snapshot updates increase forgetting by 0.0285, 95% CI 0.0034–0.0667.
- The historical GRAD-192 full encoder is 96.63±1.41, 63.80±2.71, and 87.30±4.94 on Decomp/MCP/MRT (Table 23), which is not the same configuration as the final 512-character batch.
What are some points that can be further explored?
The authors state that detection depends on observable records, new environments require annotation and mapping, and gains on held-out attack families depend on the endpoints.
Limitations and future directions noted by the authors
- GRAD relies on observable execution records; incomplete logs or out-of-context information limit detection (Section 6).
- New environments require annotated data and a mapping to graph attributes and candidate relations (Section 6).
- CAA brings growing snapshot storage, and the benefit of leaving out attack families depends on the endpoint (Section 6).
- The authors state that future work will address coverage, annotation efficiency, and expert merging (Section 6).
- The current dataset contains execution histories of a given length; longer sessions increase graph size and the evidence that must be retained. Incremental graph processing and selective history retention are directions for evaluating this scaling. Integrating other agent frameworks also requires mapping tool and memory events to the input schema (Appendix J).
- Continuous evaluation depends on task flows, training allocation, expert initialization, and routing policy; expanding the expert library increases storage and selection costs. Runtime evaluation depends on attack signals, agent configuration, and intervention behavior. Frozen-detector transfer and recalibration answer different deployment questions (Appendix J).
Experimental coverage
- The static main comparison is Decomp, MCP, MRT, with 174/58, 114/118, 50/50 per seed evaluation unit, and fold counts of 1, 5, 5; Table 1's GRAD aggregates three seeds (Appendix A, Table 1).
- The decision-moment comparison uses the original Decomp fold0, with 232 test cases and 58 base-case groups. Appendix P additionally checks a frozen checkpoint with 47 newly generated clusters and 188 cases; this check did not retrain CAA (Section 4.2, Appendix P).
- The closed-loop backbones are DeepSeek Flash and GLM-5.3-Flash; Table 6 has 20 attacks and 20 benign runs per arm, four attack families, 240 runs in total. The judge for Sequential Monitor is DeepSeek Flash, with a threshold of 0.5 (Section 4.5, Appendix F).
- CAA uses five seeds and two task orders. The cross-dataset stream is a sample-level split of Decomp/MCP/MRT, and attack families may repeat across training and evaluation; there is also a family-isolated mechanism stream (Section 4.4, Appendix L).
- The authors state that MRT adaptation of Sequential Monitor was not performed. The P1/R1 evaluation of AgentDojo evaluates undefended DeepSeek Flash, not GRAD: P1 has an overall ASR of 1.43% over 70 attacks, and R1 has an ASR of 0.00% over 94 records. The paper does not report graph construction time in the Table 7 timings, nor the monitoring token count and peak detector memory for this batch (Appendix B, C, G).
Summarize the main content of the paper
GRAD performs risk detection on execution graphs and adapts to tasks with snapshot experts, with an AUROC of 97.19% on Decomp.
GRAD is an execution-graph detection and continual adaptation framework for LLM agents, targeting memory poisoning and decomposed attacks that spread malicious evidence across multiple operations.
Request-level defenses do not link earlier information to later actions. The authors use a typed execution graph to jointly encode operation text, event attributes and candidate relations, and output graph-level risk. CAA adds task experts on frozen backbone snapshots, and at inference selects one expert by node type and graph size. A pre-execution gate uses a separately trained detector for local inference; if the development-set threshold is reached, it blocks and replans without calling a monitoring API.
- Table 1: On Decomp, MCP and MRT, GRAD's AUROC is 97.19%, 77.59% and 86.60% (three seeds). The comparison rows are locally adapted components; the Sequential Monitor's MRT adaptation was not run.
- Table 3 decision time: GRAD is 97.0%, Temporal is 91.7%, Text-only is 86.4%. On the 47 new clusters in Appendix P, GRAD is 0.615 and Temporal is 0.675; the authors state that the original advantage did not reproduce.
- Table 4: On Decomp both single channels are below 87%; on MCP text-only is 79.89%, higher than GRAD's 77.59%.
- On the cross-dataset stream, CAA's final accuracy is 77.86%, 13.08 percentage points higher than Replay's 64.78%; after equal weighting it is 9.05 percentage points. On the mechanism stream after equal weighting, snapshot experts are 47.5% and Replay is 53.5%.
- Table 6: On DeepSeek Flash, GRAD's ASR/BU/UA is 0/75/80, and on GLM-5.3-Flash it is 10/40/30; the monitor in the same table is 0/50/0 on GLM. On DeepSeek trajectories, the gate forward p50 is 8.4 ms (Table 7).
The authors state that GRAD combines execution-graph risk detection with a scalable snapshot expert library, and that the closed-loop results characterize the safety–utility trade-off of local blocking and replanning. CAA adds experts on updated backbone snapshots while retaining previously acquired experts. Detection depends on observable records, and gains on held-out families depend on the endpoints used.