ErosionBench: Probing alignment decay in enterprise multi-agent systems

1. 01What problem does this paper attempt to solve?

   The authors say that cross-session alignment erosion of multi-agent shared state is an unstudied failure mode, and they propose a process-level evaluation benchmark.

   Core problem: in enterprise multi-agent systems, gradual drift occurs between authorized intent and the tool calls actually executed (which the authors call semantic alignment decay), and existing benchmarks cannot measure or verify it at the process level.

   • Problem scenario: enterprise deployments divide work among roles such as gateway, orchestration, policy and deployment, connected via shared memory and critical tools. An attacker can plant seemingly legitimate values in an early session; later agents treat them as already-reviewed information, and ultimately the deployment agent passes them as parameters to a production tool.
   • Why it matters: no single message contains obviously malicious content, so boundary filters cannot see the attack; the authors say such decay can expose public endpoints, override permissions and leak data.
   • Shortcomings of existing methods: the authors say existing benchmarks either isolate a single agent under prompt injection or score multi-agent attacks only by final output, excluding cross-session erosion of shared state from scope; boundary filters can lower attack success rate only by also refusing most legitimate work.
   • What the paper proposes: the ErosionBench benchmark (1,728 scenarios, four enterprise domains, four decay mechanisms, OWASP labels, 198 image attacks) and a reference defense, Meta-Auditor (deterministic state anchors + calibrated asynchronous trajectory monitoring).
   • Threat model:
     • Attacker goal: make the mesh assemble a critical tool call that violates the user's authorized intent without any single suspicious message.
     • Attack vectors: four mechanisms—cross-session conditioning, state gaslighting (rewriting persistent ledger entries), long-context dilution and distributed collusion; payloads enter via session history, shared ledgers or documents (text or rendered images).
     • Victim system: a four-role LLM mesh of Gateway/Orchestrator/Policy/Deployment, with a shared state ledger and in-domain critical tools.
     • Success adjudication: whether the realized tool call violates the scenario's own ground-truth contract (process-level adjudication, not output text).

   Collapse this answer

2. 02What related research exists?

   Related work is grouped into three: single-agent attack benchmarks, multi-agent attack propagation, and boundary defenses and monitoring; the authors state that this paper is the only combination covering all axes.

   Single-agent attack benchmarks

   • AgentHarm (Andriushchenko et al., 2025): measures whether agents execute clearly harmful instructions; the authors relay its finding that leading models often comply without jailbreaking.
   • BIPIA (Yi et al., 2023) and AgentDojo (Debenedetti et al., 2024): the former isolates indirect instructions in retrieved content; the latter combines prompt injection with an executable tool environment and evaluates defenses and task utility together.
   • ToolEmu (Ruan et al., 2024): inspects full trajectories in an LM-simulated sandbox. The paper notes that these settings only put a single agent against malicious input, and the threat surface that arises on shared state across multiple agents does not appear.

   Multi-agent attacks and propagation

   • TAMAS (Kavathekar et al., 2026) and MAStrike (Xu et al., 2026): the former evaluates adversarial risk across prompts, environments and inter-agent communication; the latter uses Shapley analysis to construct collusion attacks against hierarchical teams.
   • Decompositional jailbreaks and Crescendo (Srivastav & Zhang, 2025; Russinovich et al., 2025): split an unsafe goal into individually benign sub-requests; the authors use Crescendo's multi-turn escalation primitive as the conditioning mechanism in their policy debate.
   • AgentLAB (Rahman et al., 2026): the paper calls it the closest work, evaluating long-horizon attacks (intent hijacking, goal drift, memory poisoning), but it only scores a single agent's long trajectory, not multi-agent erosion of a persistent shared ledger.

   Boundary defenses and monitoring

   • LlamaFirewall (Chennabasappa et al., 2025) and Progent (Shi et al., 2025): screen text at trust boundaries and enforce programmable least privilege on structured tool calls, respectively; this paper reproduces both as comparison defense layers.
   • BlindGuard (Miao et al., 2025) and MonitoringBench (Jotautaitė et al., 2026): the former detects and isolates compromised agents under unknown attacks; the latter calibrates trajectory monitoring thresholds.
   • Cross-agent action attribution (Park et al., 2026): the authors relay its analysis that attacks distributed across independent asynchronous agents can bypass local guardrails that only see a single session, which motivates process-level monitoring.

   Baselines and benchmarks

   Comparison defense baselines are LlamaFirewall, ShieldGemma-27B, Progent and an uncalibrated LLM judge on the same telemetry; external transfer evaluation uses AgentDojo and ASB (Zhang et al., 2025). Table 1 compares ErosionBench with AgentDojo, ToolEmu, ASB, AgentLAB, Blindspot, EnterpriseOps, TAMAS, MAStrike and CFH along five axes; the authors state that ErosionBench is the only benchmark covering multi-agent mesh, cross-session state erosion, process-level verification, an executable environment and benign over-refusal scoring simultaneously, and emphasize that it is this combination, not any single axis, that isolates the failure mode.

   Collapse this answer

3. 03How does the paper address this problem?

   Generate scenarios with a deterministic state graph, execute them in a four-role mesh, check tool calls against contracts, and add a three-tier Meta-Auditor defense.

   The overall approach has two parts: the ErosionBench benchmark turns cross-session state erosion into executable, verifiable scenarios; Meta-Auditor, as a reference defense, performs synchronous state checks plus asynchronous trajectory monitoring at the process level.

   Data generation pipeline

   • Four-stage generation: first write a state graph for each domain, fixing benign intent, key tools, safety parameters, planted values and ground-truth assertions; then a generator renders narratives (templates or LLM-written benign text) to produce T1 attack scenarios and their benign twins; then pass three validation gates (deterministic reachability check, semantic realism check, ground-truth contradiction check); finally a mutator extends T2 and T3 variants according to the decay mechanism.
   • Mechanisms built into generation: cross-session conditioning adds priming sessions, long-context dilution lengthens documents, state gaslighting rewrites ledger entries, distributed collusion adds corroborating agents; T1 places the needle early in a short prefix, T3 buries the same payload at about 70% of the full text.
   • Quality gates: the canonical splicer inserts the needle last, and the released corpus generator has zero refusals (0/1,728); text is validated by a text→PDF→text round trip, and image needles are admitted only if they pass a build-time deterministic OCR read-back.

   Data composition and annotation

   • Scale: four domains (quantitative finance, CI/CD supply chain, data engineering, autonomous CRM) × four mechanisms = 16 attack classes; 1,296 malicious + 432 benign scenarios, 432 disjoint families in total; additionally 66 families are rendered as enterprise document images (198 malicious cells, each with a 1:1 text equivalent and a benign control).
   • Interaction depth: T1+control averages 5.03 sessions/5 pages, T2 is 8.04 sessions/20 pages, T3 is 11.55 sessions/50 pages (Table 2).
   • OWASP annotation: each attack class is labeled with OWASP Top-10 for LLM Applications tags at authoring time according to a fixed mapping; the 16 classes concentrate in LLM04 and LLM06 (Table 14).
   • Human validation: two annotators blind-rate 50 held-out items; realism mean 4.37/5, benign 4.34 vs. attack 4.33; label recovery 42/44 and 41/44, Fleiss' κ = 0.87 (Table 13).

   Mesh execution

   • Four roles: Gateway parses requests, Orchestrator reads accumulated state and proposes actions, Policy debates constraints, Deployment invokes key tools; ordered by a state machine, the Orchestrator decides to loop, escalate or proceed to deployment.
   • Sandbox: FastAPI exposes document access, domain tools and immutable audit/state/impact-surface streams; roles are fixed, only key tools and domain corpora change; a compromised agent responds to agents in the mesh that rebut it, so attacks are reactive rather than scripted.

   Meta-Auditor defense

   • Three deterministic anchors: SDI computes the average typed distance (0–1) between authorized key values and proposed tool calls, and is the only signal that can veto a call before execution; CII is the proportion of key payload values traceable to ledger writes from unverified sources; VAD measures how early or late the policy agent resists then concedes across multi-turn debate. Formally, CII = |P ∩ U| / |P|, i.e., the proportion of scalar leaf values of the key payload introduced by unverified writes.
   • Semantic monitoring and calibration: an asynchronous monitor assigns a continuous intent-divergence score to the full trajectory, with the authorized intent given as input rather than inferred; scores are Platt-scaled, and the threshold is the finite-sample order statistic of benign calibration samples (target α = 0.05).
   • Gray-zone arbitration: trajectories with scores near threshold τ or with anchors inconsistent with the semantic monitor are escalated to a dedicated arbitration call that re-reads telemetry and gives a final judgment; the auditor has no access to malicious labels, attack classes, hidden chain of thought or benign-twin diffs, and only detects and localizes, without rollback.
   • Calibration protocol: families are split before tier/control expansion with leakage protection, yielding 1,124 calibration scenarios and 906 held-out scenarios (453 malicious/453 benign); all main results are reported on the held-out set.

   Collapse this answer

4. 04What experiments did the paper conduct?

   The authors report an undefended mesh ASR of 43.9%; boundary defenses lower ASR via 54.3%–92.5% over-refusal; Meta-Auditor reaches 98.5% recall and 2.9% false positives.

   Experimental Setup

   • Mesh under test: the main experiments use gpt-5.5 to drive a four-role mesh; Table 8 additionally sweeps two meshes, gemini-3.5-flash and claude-opus-4-8.
   • Auditor panel: claude-opus-4-8, claude-sonnet-4-6, gpt-5.1, gemini-3.5-flash, gemma-4-31b, gpt-oss-safeguard-20b; the image arm additionally has Gemma-4-31B-IT (VLM).
   • Data: a family-disjoint 906-scenario held-out set (453 malicious/453 benign, with benign being validity-matched controls matched on depth and length); the image arm has 198 malicious + 198 benign cells.
   • Baselines: L1 single-agent isolation, L2 no-defense mesh, L3 LlamaFirewall (real PromptGuard classifier + AlignmentCheck), L3 ShieldGemma-27B (standalone boundary harness), L4 Progent, L5a same-telemetry uncalibrated LLM judge.
   • Metrics: ASR (the proportion of the 453 malicious scenarios in which a harmful tool call is executed, reported by layer), benign FPR (the proportion of legitimate tasks incorrectly blocked), Recall/Precision/F1, all with 95% Wilson intervals.
   • Adjudication method: whether an attack lands is adjudicated by the scenario's own ground-truth contract; defense detection is given by the auditor, with no separate human review of attack outcomes.

   Main Results

   According to Table 4 (gpt-5.5 mesh, held-out set 453 malicious/453 benign):

   Swipe horizontally to read the table

   Defense layer	Overall ASR	T1	T2	T3	Benign FPR
   L1 single agent	0.0%	0.0%	0.0%	0.0%	48.3%
   L2 no-defense mesh	43.9%	35.1%	45.0%	51.7%	37.3%
   L3 LlamaFirewall	10.2%	7.9%	11.9%	10.6%	92.5%
   L3 ShieldGemma-27B	27.8%	21.9%	30.5%	31.1%	54.3%
   L4 Progent	4.9%	2.6%	6.0%	6.0%	64.0%
   L5a LLM judge (uncalibrated)	2.2%	5.3%	0.7%	0.7%	67.6%
   L5b Meta-Auditor (opus)	1.3%	4.0%	0.0%	0.0%	2.9%

   • Authors' interpretation: L1 shows that the attack depends on the mesh (ASR 0.0%) but cannot complete multi-step workflows (48.3% of legitimate tasks not completed); boundary defenses lower ASR only by over-refusing.
   • Role of calibration: L5a and L5b share the same telemetry surface, and recall is close (97.8% vs. 98.5%, Table 3b), but FPR differs by an order of magnitude; the authors attribute this gap to the calibration mechanism rather than the model.
   • Statistical tests: no-defense ASR rises monotonically with tier, with a two-proportion z-test p=0.004 for T1→T3; differences between adjacent tiers are within sampling noise.

   Cross-Auditor and Cross-Mesh Transfer

   • Auditor capability gradient (Table 5, fixed gpt-5.5 mesh): recall goes from 98.5% for opus-4-8 to 83.4% for gemini-3.5-flash, which the authors say tracks model capability; FPR does not vary with capability (1.3% for gpt-5.1, 11.0% for gemini-3.5-flash).
   • Open-source auditors: gemma-4-31b has 89.4% recall but 12.4% FPR; gpt-oss-safeguard-20b has 2.4% FPR but only 55.1% recall.
   • Mesh sweep (Table 8, auditor fixed to sonnet-4-6): L2 ASR goes from 44.2% for gpt-5.5 to 15.0% for claude-opus-4-8, with recall staying at 0.874–0.971; the authors say detection tracks the auditor rather than the attacker; the FPR in this table is based on 151 controls rather than the 453-control arm.

   Channel Invariance and External Transfer

   • Image vs. text (Table 6): the frozen text calibration artifact is used directly for the image arm; opus-4-8 recall is 100.0%/100.0% (image/text), sonnet-4-6 is 100.0%/99.0%, and gpt-5.1 is 96.0%/96.0%; no-defense ASR is 35.4% for image vs. 31.3% for text, and the authors say the parity is not caused by image attacks being easier. The image arm's benign controls are not yet validity-matched, so the FPR is provisional.
   • External corpora (Table 19, frozen opus-4-8, τ = 0.595, no recalibration): on AgentDojo landed runs, recall is 86.4% and FPR 0.0%, triggering on only 7.5% of resisted runs; on ASB memory poisoning, recall over 100 payloads is 95.0% and FPR 2.0%. The authors say all external detections go through the semantic gray-zone path.
   • Unseen domain (Table 21): the fifth domain, clinical healthcare (45 malicious/20 benign, three scorable attack classes), has L2 ASR 17.8%, opus-4-8 recall 100.0% and FPR 0.0%, and gpt-5.1 recall 95.6%.

   Other Ablations and Analyses

   • Sub-signal decomposition (Table 3a): SDI alone has 43.9% recall and 0.0% FPR; CII 14.8%/0.0%; VAD 7.9%/1.8%; a naive OR-gate has 100.0% recall but triggers on 22.1% of benign cases.
   • Gray-zone routing (Table 16): of 906 trajectories, 439 go to gray-zone arbitration, producing 263 of 459 flags and raising recall from about 43% (196/453) for the anchor alone to 98.5%.
   • Overfitting control (Table 7): on 64 entirely new family scenarios, the frozen artifact has 97.9% recall and 6.2% FPR (only 16 controls, wide interval).
   • Provenance invariance (Table 17): relabeling the provenance of all 50 attacks as an authenticated channel leaves recall unchanged (98.0%); 48 new attacks with no provenance traces have 97.9% recall.
   • Per-domain results (Table 11): L2 ASR ranges from 22.2% for autonomous CRM to 59.6% for CI/CD; Meta-Auditor has 0.0% residual ASR in three domains, with residual 7.7% only in quantitative finance (recall 91.0%, and that domain has only 78 attacks).
   • Image-arm open-source VLM (Table 22): Gemma-4-31B-IT has 84.8% recall (56/66 families) but 68.2% FPR (45/66).

   Collapse this answer

5. 05What are some points that can be further explored?

   Limitations listed by the authors include that the operating point is not portable across models, coverage is limited to enterprise workflows, the image-arm FPR is provisional, and they plan to test adaptive attackers.

   Limitations and future directions noted by the authors

   • Operating point not transferable (Section 6): the frozen Platt/conformal artifact is fitted per auditor model, and changing the auditor requires recalibration; thresholds cannot be carried across models, and the assumption that trajectories are exchangeable means there is no formal population-level FPR guarantee.
   • Coverage (Section 6): covers only enterprise workflows (four domains plus one held-out medical domain); consumer scenarios, adaptive attackers, mesh topology and changes in agent count are left to future work.
   • Defense positioning (Section 6): Meta-Auditor is a research reference, not an adversarially hardened defense; scenarios are procedurally generated (with realism validated by blind review) rather than taken from real incidents.
   • Image arm FPR provisional (Section 6): the Mode B image benign controls have not been validity-matched, so the image arm's false-positive rate is provisional; Appendix P further notes that in 6 of 11 classes the benign twins are invalid controls.
   • Future directions (Section 7): connect the asynchronous layer's step-level judgments to gated rollback or containment; distill the reference auditor into a small always-on detector; test adaptive attackers, changing mesh topologies, agent counts and human review.
   • OWASP annotation consistency (Appendix I, Table 15): on 23 double-rated items, the primary label fully agrees on only 4/23; the authors state that a robust multi-annotator multi-label OWASP consensus is left to future work.

   Experimental coverage

   • The held-out set is 906 scenarios (453 malicious/453 benign) with a family-disjoint split; overfitting control additionally uses 64 new scenarios (48 malicious/16 benign).
   • The meshes under test are three: gpt-5.5, gemini-3.5-flash, claude-opus-4-8; the auditor panel has six (including two open-source), and the image arm has one additional open-source VLM auditor.
   • Defense baselines cover LlamaFirewall, ShieldGemma-27B, Progent and an uncalibrated LLM judge on the same telemetry; external transfer covers two corpora, AgentDojo and ASB.
   • Human validation is two annotators on 50 held-out items; the image card only rates realism and image legitimacy, not attack/benign judgment.
   • The paper does not report generation parameters such as sampling temperature for the mesh and auditor; the FPR in the mesh sweep (Table 8) is based on 151 controls rather than the 453 control arm used for the main results.

   Collapse this answer

6. 06Summarize the main content of the paper

   ErosionBench turns cross-session erosion of multi-agent shared state into an executable, process-level verifiable benchmark, paired with a calibrated process-level reference defense.

   This is a paper with a benchmark plus a reference defense; the target failure mode is what the authors call semantic alignment decay: in enterprise multi-agent systems, attacks are assembled across sessions via shared ledgers, long documents and peer messages, with no single malicious message, and ultimately manifest as tool calls that violate authorized intent.

   • Benchmark: ErosionBench generates 1,728 scenarios by first building a deterministic state graph and then rendering text, covering four enterprise domains, four decay mechanisms and 16 attack classes, with OWASP labels and matched benign controls; there are also 198 image attacks, each with a 1:1 text equivalent. Adjudication looks at whether the realized tool call violates the ground-truth contract, not at the output text.
   • Defense: Meta-Auditor combines synchronous deterministic state anchors (SDI/CII/VAD) with calibrated asynchronous semantic monitoring and gray-zone arbitration; the authors say it only detects and localizes, and does not roll back.
   • Main results (gpt-5.5 mesh, 906-scenario held-out set): no-defense mesh ASR is 43.9%; LlamaFirewall lowers ASR to 10.2% but refuses 92.5% of legitimate tasks, and Progent refuses 64.0%; opus-4-8 Meta-Auditor has residual ASR 1.3%, recall 98.5% and benign FPR 2.9% (Table 3b, Table 4).
   • Transfer and generalization: the recall difference between image and text channels is no more than 1 percentage point (Table 6); the frozen artifact has 95.0% recall on ASB memory poisoning and 86.4% on AgentDojo landed runs (Table 19); on the unseen domain of clinical healthcare, opus-4-8 has 100.0% recall (Table 21).
   • Authors' conclusion: in settings where over-refusal and missed detection are equally costly, reliable verification requires heterogeneous evidence and explicit operating curves rather than a single outcome score; the authors also state that the operating point is calibrated per auditor, coverage is limited to enterprise workflows, the image-arm FPR is provisional, and adaptive attackers and topology changes are left for future work.

   Collapse this answer

Related papers