Stateful online monitoring captures distributed agent attacks

Core problem: Standard safety monitors analyze only a single agent context and cannot identify distributed attacks that split harmful tasks across multiple user accounts and contexts.

• Scenario and importance: Language models can already find thousands of high-severity software vulnerabilities, and agents are being misused for cyberattacks. Attackers often split malicious tasks into seemingly benign subtasks distributed across multiple accounts for execution, making a single conversation record appear harmless.
• Shortcomings of existing methods: Current standard monitors score only one agent context at a time and are architecturally unable to discover misuse that is visible only at the aggregate level.
• Core observation: Subtasks of a distributed attack share subtle features derived from the attack objective in embedding space; even if the attacker obfuscates each subtask, these patterns can still be recovered through clustering.
• Paper's approach: Propose an online stateful monitor that collects weak suspicious signals across accounts through real-time streaming clustering, escalating to a language model for cross-context misuse determination only in rare cases.
• Threat model:
  • Objective: Use agents to complete harmful cybersecurity tasks.
  • Knowledge: The attacker knows the monitor exists and attempts to evade detection through distributed splitting.
  • Capability: Can create arbitrary new accounts and distribute subtasks across different agent contexts.
  • Victim system: Service providers deploying language model agents and their users.

Single-context decomposition attacks

• Li et al. (2024) and Yueh-Han et al. (2025): perform decomposition attacks within a single model context, splitting harmful tasks into sequential subtasks. The paper notes that these works are designed for non-agentic coding tasks and are not distributed across user accounts.
• Glukhov et al. (2024), Jones et al. (2025) and Brown et al. (2025): decompose harmful tasks in simple question-answering settings.

Multi-agent systems and cybersecurity

• Lin et al. (2025): propose the ARTEMIS multi-agent framework for completing cybersecurity tasks. This paper builds distributed attacks on this framework but spreads subtasks across different user accounts.
• Zhang et al. (2024, 2025): propose the Cybench and BountyBench cybersecurity benchmarks, respectively, for evaluating agent cyber capabilities.

Safety monitoring and defenses

• Sharma et al. (2025), Cunningham et al. (2026) and Kramár et al. (2026): train and deploy safety monitors, but the paper notes that these standard monitors are architecturally unable to identify distributed misuse.
• Inan et al. (2023): propose Llama Guard, which uses a language model to judge the safety of inputs and outputs.

Streaming clustering and retrieval

• Aggarwal et al. (2003), Cao et al. (2006) and Hahsler and Bolaños (2016): propose data stream clustering algorithms for maintaining compact clusters online. This paper uses these techniques to match new transcripts to online clusters in real time.
• Tamkin et al. (2024): propose Clio, which analyzes real AI usage data through embeddings and clustering, but the paper notes that its periodic re-clustering approach is too slow and costly for real-time monitoring.

Baselines and benchmarks

• Baseline methods: directly prompting gpt-5.2, the baseline ARTEMIS framework, sequential decomposition attacks.
• Benchmarks/datasets: Cybench, BountyBench, SWE-bench, tau2-bench, Terminal-bench, WebLINX.

The authors position the distinction of this paper from related work as: the first realization of fully general distributed agent attacks across user accounts, and the proposal of an online stateful monitor to detect such attacks in real time.

The paper proposes an Online Stateful Monitor, which models distributed abuse detection as a streaming clustering problem, maintaining compact online clusters to match new transcripts in real time, and invoking a language model for cross-context reasoning only in rare cases.

Problem Setting and Formalization

• The defender receives a stream of agent requests from tens of thousands of users and must classify each new query and agent action as benign or harmful.
• Attackers can create arbitrary new accounts and split harmful tasks into seemingly benign subtasks executed in a distributed manner.
• The standard monitor is a language model judge that uses only the current agent context x_t to produce a 0-1 suspiciousness score.

Online Clustering to Retrieve Relevant Context

• For each new request, the standard monitor first provides a suspiciousness score, and then a Voyage AI embedding model is used to compute the embedding e_t of the current context.
• e_t is compared with existing cluster centroids to find the nearest centroid j^⋆ = argmax ⟨ e_t, c_j ⟩. If the similarity is below a threshold τ_route, a new cluster is opened; otherwise the nearest cluster centroid is updated online.
• When agent contexts are repeatedly assigned to the same cluster, these clusters are merged. Each cluster maintains a buffer sorted by suspiciousness that stores deduplicated prior suspicious contexts.

Cross-Context Monitoring and Escalation Mechanism

• The cross-context monitor is invoked in two cases: when the current request's suspiciousness is high but insufficient to directly reject; or when cross-session evidence from different user accounts has accumulated in a cluster.
• To flag a cluster, the largest k suspiciousness scores in the buffer are summed, and if the sum exceeds a threshold calibrated on held-out data, a review is triggered. It is also required that at least m prior contexts exceed a specific score.
• Algorithm 1 describes the complete pipeline: standard monitor scoring → embedding → routing to the nearest cluster or creating a new cluster → updating the buffer → if escalation conditions are met, assembling attack evidence and invoking the cross-context monitor.

Cost Model and Latency Analysis

• The per-request cost is T_t = O(1) + O(dM_t + M_t log M_t) + q_t C_cc, where O(1) is the standard monitor cost, dM_t is the centroid dot-product cost, M_t log M_t is the sorting cost, C_cc is the cross-context monitor cost, and q_t ∈ {0,1} indicates whether the request is sent.
• Only a small fraction of requests incur language model inference cost; most requests incur only local scoring and online routing cost.

Success Determination and Evaluation Method

• A refusal policy is used, treating exploitation tasks in the CTF benchmark as disallowed while allowing ordinary software engineering and defensive security work.
• All monitor judgments use the same cybersecurity prompt and the gpt-4.1-mini-2025-04-14 model.
• Evaluation is performed by simulating data center traffic, inserting distributed attacks into benign agent traffic, with benign trajectories sampled at a 10x or 100x ratio.

Experimental setup

• Model under test: gpt-4.1-mini-2025-04-14 (used for both the standard monitor and the stateful monitor).
• Attack models: gpt-oss-120b as supervisor, gpt-5.2 as sub-agent; GLM-5 as a strong-supervisor baseline.
• Datasets/benchmarks: Cybench (subset of 10 hard tasks), BountyBench; benign background comes from coding agent and web chat trajectories such as SWE-bench, tau2-bench, Terminal-bench, WebLINX.
• Baselines: directly prompting gpt-5.2, the baseline ARTEMIS framework, sequential decomposition attacks.
• Metric definitions: campaign recall (recall if any harmful request is refused), benign conversation FPR, TTD (number of attacker turns before a harmful turn is detected; TTD is 0 if the first harmful turn is refused; missed campaigns are counted as the trajectory length).
• Review method: all monitoring judgments are made by gpt-4.1-mini-2025-04-14 using the same cybersecurity prompt.
• Sample size and repetitions: 50 distributed attack tasks, with new benign background samples repeated 5 times; the red-team exercise produced over 100,000 attacks, involving 2,985 sessions and 1,159 user IDs.

Main results

Swipe horizontally to read the table

Condition	Campaign recall	Benign conversation FPR	TTD99	Source
Stateful monitor (10x background)	81.1–87.2%	2.35–3.09%	not reported	Figure 5
Standard monitor (10x background)	75.0%	2.72%	not reported	Figure 5
Stateful monitor (100x background)	65.8%	1.47%	25.4	Figure 5, 6
Standard monitor (100x background)	55%	1.47%	37.2	Figure 5, 6
Stateful monitor (100x background, high recall)	85.0%	4.30%	25.4	Figure 6
Standard monitor (100x background, same recall)	85.0%	9.01%	37.2	Figure 6

• The authors claim the stateful monitor's operating points are better than the standard monitor's on the Pareto frontier.
• Under 100x benign background, the stateful monitor has 10.8 percentage points higher recall than the standard monitor at the same FPR.
• The stateful monitor reduces TTD99 from 37.2 to 25.4, and requires a lower FPR to achieve the same recall.

Red-team exercise

• What was tested: six red-team members white-box tested the monitor over two months, attempting exploitation tasks in BountyBench and CyBench.
• Results: the defense captured 528/644 (82%) of monitor context poisoning attacks and 29/29 sequential decomposition attacks. Figure 8 shows that as cross-session correlated activity accumulates, the stateful monitor's flagging rate rises for both distributed attacks and semantic-cover jailbreaks.
• Authors' interpretation: adaptive attackers reuse attack variants across sessions, so the stateful monitor is also effective against standard jailbreaks.

Other ablations and analyses

• Red-team strategies in embedding space (Figure 7): four attack strategies are identified: monitor context poisoning, sequential decomposition, distributed agent attacks, semantic-cover jailbreaks.
• Scalability (Figure 9, 10): routing cost follows a linear model rather than a quadratic model; peak state size grows linearly with the peak number of clusters.
• Attack capability (Table 1): gpt-oss-120b's direct solve rate is 0.000, but when acting as supervisor guiding gpt-5.2 sub-agents the solve rate is 0.367–0.400; with GLM-5 as supervisor it reaches 0.900–1.000.
• Latency (Figure 4): the stateful monitor's latency is close to the standard monitor's on 99% of traffic, with increased latency only in the P99 tail.
• Failure modes: renaming files or using different vocabulary can split attacks into neighboring clusters to delay review; ambiguous suspicious examples may persist too long in cluster memory, causing high false positive rates.

Limitations and future directions noted by the authors

• Scale of benign background traffic: The authors state that when benign background traffic becomes very large, the detection advantage still exists but shrinks (Section 4).
• Adaptive attackers: The authors note that adaptive attackers who work hard to obfuscate tasks or use strong jailbreaks can still hide evidence and partially complete tasks (Section 6).
• Embedding model robustness: Red teaming found that renaming files or using different vocabulary can deceive the embedding model, splitting attacks into neighboring clusters to delay review (Section 5).
• False positive rate: Ambiguous suspicious examples may persist too long in cluster memory, leading to a high false positive rate (Section 5).
• Distributed abuse unresolved: The authors explicitly state that the stateful monitor does not solve the problem of distributed abuse (Section 6).

Experimental coverage

• The model under test is gpt-4.1-mini-2025-04-14, 1 in total.
• Attack models include gpt-oss-120b and GLM-5 as supervisors, and gpt-5.2 as subagents.
• Defense evaluation includes two scales of benign background traffic: 10x and 100x.
• Each condition is repeated 5 times, using 50 distributed attack tasks.
• The red-team exercise includes 6 participants over 2 months, producing more than 100,000 attacks.
• The paper does not report specific latency overhead figures for the cross-context monitor.

The paper addresses the problem that standard safety monitors cannot identify cross-account distributed agent attacks, and proposes an online stateful monitor. The monitor collects weak suspicious signals across accounts through real-time streaming clustering, and escalates to a language model for cross-context review only when evidence accumulates to a threshold.

• Distributed attack construction: Based on the ARTEMIS multi-agent framework, a gpt-oss-120b supervisor splits harmful cyber tasks into seemingly benign subtasks and distributes them to gpt-5.2 subagents for execution. On the Cybench hard subset, this attack achieves a solve rate of 36.7–40.0%, whereas gpt-oss-120b's direct solve rate is 0%.
• Monitor design: Uses the Voyage AI embedding model to embed agent contexts and cluster them online, routing by comparing new requests with cluster centroids, and invoking the cross-context monitor only for clusters with sufficient accumulated evidence.
• Detection performance: Under a 10x benign background, the stateful monitor achieves 81.1–87.2% activity recall with an FPR of 2.35–3.09%; under a 100x background, recall is 65.8% with an FPR of 1.47%. The authors state this detects attacks 30% earlier than standard monitors.
• Latency performance: For about 99% of user traffic, the stateful monitor's latency is close to that of the standard monitor, with only P99 tail latency increasing.
• Red-team exercise: During two months of white-box testing, the defense captured 82% of monitor context poisoning attacks and all 29 sequential decomposition attacks. The authors found the stateful monitor is also effective against standard jailbreaks, because attackers reuse attack variants across accounts.
• Authors' conclusion: The stateful monitor represents a new class of safety monitor that can reason across user populations rather than isolated transcripts, but it does not fully solve the problem of distributed abuse.