From Spark to Wildfire: Modeling and Mitigating Error Cascades in LLM-Based Multi-Agent Collaboration

LLM multi-agent collaboration can solidify local atomic errors into system-level false consensus through contextual reuse.

• Scenario: the authors state that division of labor is assumed by default to improve reliability, but when context is recursively reused, small deviations in factuality or faithfulness are repeatedly cited and converge into a collective consensus that contradicts task evidence. Figure 1 depicts this process leading to safety breaches or operational failures.
• Existing gaps: the authors state that work on prompt injection, jailbreaks, and RAG truthfulness stops at single agents, lacking a direct characterization of collaborative amplification; adding critics to MAS or changing collaboration patterns interferes with information flow and treats errors as static anomalies to be filtered.
• Hypothesis: after writing the message flow as a directed dependency graph, context reuse drives trackable contagion rather than random diffusion; whether early amplification occurs can be checked from structure and propagation and correction parameters.
• Proposed content: propagation dynamics and an early-risk criterion, three classes of endogenous vulnerabilities across six mainstream frameworks, single-seed targeted consensus corruption, and a lineage-graph governance layer (message-layer plugin) that does not change communication structure.
• Threat model:
  • Assets and success: protecting the integrity of collaborative consensus. An exogenous attack succeeds if it raises the false consensus proportion S(t) within an interaction budget, or makes final task correctness lower than the undisturbed baseline.
  • Knowledge: gray-box, knowing graph G and interaction history, without model weights; generalizable to black-box, inferring functional topology only from interaction trajectories.
  • Capability boundary: application-layer interfaces only, including user prompts, untrusted retrieved documents, and injected messages. Cannot modify hidden instructions, cannot execute code on the host, no training poisoning; limited to inference phase and bounded budget. The system-instruction injection in Section IV-A is for measurement only and the authors state it is not part of this capability.
  • Victim: the final decision or generated artifact of the LLM-MAS. There are also non-adversarial endogenous errors, with seeds from decoding, limited context windows, and hallucination.

Related work is in Section IX, and the introduction separately discusses defenses of single-agent verification and architecture modification.

Adversarial attacks

• Communication surface: Surveys by Ferrag et al. (2025) and Kong et al. (2025) organize prompt injection, context hijacking, and protocol-layer manipulation.
• Cross-round propagation: Malicious outputs can propagate along downstream dependencies; indirect prompt injection can pass through multi-round delegation, overriding system intent and amplifying beyond the initiating agent.
• Consensus steering: Biasing one agent or the coordination channel can turn cooperative verification into an amplification path. The text also mentions intercepting inter-agent messages, and combinations of web pages, tools, and shared memory. The authors state these observations support orchestration-layer governance rather than ad hoc per-agent defenses.

Failures without an adversary

• Reliability: Pan et al. (2025) report limited reliability gains from chained agents and multi-agent-specific collapses; more than 1,600 trajectories were classified into 14 failure modes, falling under system design, inter-agent mismatch, and task verification failure.
• Symptoms: Role misunderstanding, overwriting each other's context, ineffective loops, and forgetting earlier constraints in long interactions.
• Error reinforcement: Without external verification, shared errors are reinforced; the authors use SagaLLM's parallel booking conflict as an example, noting the lack of rollback and global state checking.

Defenses and detection

• Individual guardrails: Structured prompts, output filtering, and policy tools only constrain a single agent. The authors state these cannot see cross-agent dependencies and will miss inconsistencies after messages and tool calls pass through the team.
• System monitoring: Verifier agents, watchdogs, SentinelAgent's interaction graph plus LLM monitoring, and Audit-LLM's log auditing.
• The authors' judgment: Inter-agent links remain under-protected; multi-hop attacks can evade detectors, adversarial steering weakens consensus reliability, and monitoring overhead limits deployment. The directions summarized in the text are orchestration-layer trust policies, unified logging, supervised pauses or rollbacks, least privilege, and communication provenance.

Experimental baselines and benchmarks

• Baselines: Reflection (self-check and self-correct before sending), AGrail (checks each outbound message), CFG (checks against acceptable edges, tries another legal route on rejection, inspired by ControlValve).
• Scenarios and frameworks: QUANT (UCI), RIGID (MATH), MMLU; LangChain, MetaGPT, AutoGen, CAMEL, CrewAI, LangGraph.

The authors position this paper as a message-layer intervention that does not change the adjacency matrix A, and use graph dynamics to separate transmission (β and ρ(A)) from correction (δ). The introduction argues that doing only single-agent verification or modifying the collaboration structure does not match the propagation of errors as a dynamic flow.

Propagation dynamics writes collaboration as a directed dependency graph, then a lineage-graph governance layer intervenes only on the message path without changing communication structure.

Problem Formulation

• Graph: G=(V,E), aij=1 means j's output enters i's subsequent context. Track a single atomic error m.
• Two error types: a factual error does not hold relative to dataset ground truth, evaluation oracle, or task-specified external references (including verified tool outputs, gold labels). A faithfulness error is defined only when there is an evidence bundle Ei(t), meaning the claim is not supported by or contradicts that evidence. Surface restatement is not a sufficient test; explicit rejection or negation does not count.
• Adoption: Xi(t)=1 means m becomes a semantic commitment or functional premise, including direct entailment, and cases where removing or negating m from context makes the conclusion or action not hold. Surface restatement is not a sufficient test; explicit rejection or negation does not count.
• False consensus: si(t) is the adoption probability. Coverage is S(t)=1/n∑_i=1ⁿs_i(t). When S(t) exceeds a preset threshold τ and remains above it for several consecutive rounds, it is recorded as error lock-in. The numeric value of τ is not given in the paper.

Graph Dynamics and Risk Criterion

The one-step update is s_i(t+1)=(1−δ)s_i(t)+(1−s_i(t))f_i({s_j(t)}_{j∈ N(i)},G). δ aggregates forgetting, self-correction, and external verification. The default fi is the product form of round-independent cascades: each active upstream neighbor has one opportunity per round, and one success triggers adoption, with probability β. The Poisson form is used only when a continuous-time interpretation is needed.

When early si is much less than 1, the amplification condition is βρ(A) greater than δ; the auxiliary metric R is approximately βρ(A)/δ. When δ is near 0, R is ill-conditioned, and the authors instead use the inequality as the primary check, with R only diagnostic. The authors state the product form better matches one attempt per round and separates the agent-side β from the structure A. Appendix A fits β and δ using neutral codewords. The authors state the fitted curves in Figure 3 largely fall within the observed uncertainty band; positional text in the figure is unreadable.

Positioning of the Three Endogenous Vulnerability Classes

• Cascade amplification: when early correction is weak, simultaneous mention by multiple neighbors compounds rather than cancels. If R is greater than 1, a single seed can expand coverage along the mode corresponding to the spectral radius. Measurement writes a tracer seed into the entry agent's system instruction; the authors state this is not part of the attack capability in Section III.
• Topology sensitivity: early growth follows the principal eigenvector; the most dangerous single point is the node with the largest component of that vector. In stars, the hub has the largest component. Impact Factor is the ratio of hub-driven infection rate to leaf-driven infection rate.
• Consensus inertia: intermediate artifacts crystallize errors into constraints that later steps depend on. Accumulated Polluted Rounds counts the cumulative amount of erroneous context before intervention.

Exogenous Attack Pipeline

Injection is fixed at t=0 with one seed, used to measure an upper bound on downstream exposure; the authors state this remains within the Section III capability. The three steps are written only at the mechanism level: construct an atomic error compatible with the task format and sufficient to change downstream decisions; package it with Compliance (authoritative tone) or Security FUD (framing the seed as handling a threat that does not exist); gray-box targets high-impact nodes aligned with ρ(A), black-box targets aggregation, summarization, or final-decision roles inferred from information flow. The paper does not mark which knowledge setting each cell in the results tables belongs to.

Lineage-Graph Governance Layer

The middleware intercepts outbound messages; A remains unchanged. New claims are untrusted by default. Trusted anchors are only confirmed claims in the lineage and external evidence from designated verification tools.

1. Messages are split into independently verifiable atomic claims, classified as factual or faithfulness, then compared against the confirmed lineage: entailed claims are Green and passed; contradictory claims are Red and blocked; otherwise Yellow.
2. Yellow has three tiers: Low-Intervention does not verify, only adds an uncertainty marker and does not promote into trusted context; Balanced performs full verification on Yellow from functional hubs, with the rest same as low intervention; Strict verifies all Yellow. The operating points in the experimental tables are named Speed, Balanced, Strict. The paper does not state whether Speed and Low-Intervention are the same policy.
3. After verification: true claims are promoted to Green, false ones changed to Red with rollback evidence attached, undecided ones remain Yellow. When Red appears, the message is rewritten and retried up to a limit K, which the paper does not give. If still failing, circuit breaking: persistent Red is dropped, persistent Yellow is passed with a high-risk marker but does not enter the confirmed lineage.
4. Online intervention shares context; offline replay of logs does attribution and can approximate S(t) without affecting execution. Decomposition uses the FActScore protocol and GPT-4o-mini; entailment uses DeBERTa-v3-small; the verification layer uses a self-built knowledge base and GPT-4o-mini.

Experimental Setup

• Frameworks: Chain is LangChain, MetaGPT; Mesh is AutoGen, CAMEL; Star is CrewAI, LangGraph. One defense per topology is selected; Table V uses MetaGPT, AutoGen, LangGraph.
• Tasks: 20 questions each for QUANT, RIGID, MMLU. The appendix states QUANT uses UCI Adult Income, requiring a remote CSV and Python toolchain; RIGID requires MathAPI and is described as AMC 10/12 style; MMLU is changed to retrieval-based, with subsets High School Physics, Chemistry, Biology, one target fact per question, verified via the Wikipedia API.
• Attacks and Repetitions: Application layer, single seed, fixed t=0. Strategies are Baseline, Compliance, Security FUD. Attacks are 20×3 per cell; defenses are 20×2 per cell. The endogenous experiments additionally write a tracer seed into the entry agent's system instruction.
• Defenses and Baselines: NONE; governance layer Speed, Balanced, Strict; Reflection, AGrail, CFG. Decomposition uses GPT-4o-mini and FActScore, entailment uses DeBERTa-v3-small, verification uses a self-built knowledge base and GPT-4o-mini.
• Metrics: ASR is the proportion of tasks whose final artifact is flagged as infected; BICR is defined as 1−ASR; Safe Completion requires the artifact to be usable and uninfected. Token/Safe and Latency/Safe are averaged only over safe completions. Labels are computed by a deterministic program, not an LLM judge. S(t) is also recorded.
• Not stated in the paper: backbone dialogue models for each framework, τ, K, significance tests. Table X writes AutoGen, CAMEL as 3 agents; Table XI writes 4.

Main Results

The table below is based on Table IV, in units of ASR (%). M, Q, R are MMLU, QUANT, RIGID. Base., Comp., Sec. are Baseline, Compliance, Security FUD. Only some cells are shown due to column limits.

Swipe horizontally to read the table

Framework	Topology	Base. R	Comp. M	Sec. M	Sec. Q
LangChain	Chain	0.0	95.0	100.0	100.0
MetaGPT	Chain	46.7	100.0	96.7	76.7
AutoGen	Mesh	0.0	100.0	98.3	100.0
CAMEL	Mesh	0.0	100.0	100.0	98.3
CrewAI	Star	3.3	46.7	46.7	43.3
LangGraph	Star	11.7	100.0	100.0	98.3

• Authors' summary: intent-hiding packaging is more effective than verbatim insertion. The main text states Baseline is almost all near 0, rising only to 46.7% on MetaGPT–RIGID; the table also shows LangGraph's Baseline R at 11.7 and LangChain's Baseline M at 3.3.
• Authors' stated ranges: outside CrewAI, Compliance ASR is mostly 85.0%–100.0%, and Security FUD is 76.7%–100.0% in these settings. In Table IV, CrewAI's Compliance is 46.7, 51.7, 31.7, and Security FUD is 46.7, 43.3, 33.3.
• Same-topology differences: the authors state that on mesh, AutoGen and CAMEL, and on star, LangGraph, are near saturation under intent hiding, while CrewAI is lower, and argue that implementation-level coordination and routing affect how errors are reused.

Endogenous Vulnerabilities

• Cascade: Table X final infection rates: MetaGPT, LangGraph, CrewAI, AutoGen, CAMEL at 100.0% (SD 0.0%), LangChain at 89.2% (22.1%). The authors state five frameworks reach 100%, including settings with reviewers or QA, and role division alone does not stop propagation. The main text states S(t) rises gradually on chains, jumps on stars after hub adoption, and states LangGraph coverage is already high by t=2; mesh is nearly immediately contaminated at t≤3. Pointwise numeric values for Figure 4 are not given in the text.
• Topology: Table II: CrewAI's hub is 100.0%, leaf 15.9%, Impact Factor 6.29; LangGraph's hub is 100.0%, leaf 9.7%, Impact Factor 10.31. The authors state LangGraph's Supervisor is a strict information cut set.
• Inertia: Table III does not state the framework. At t=2, role Architect, Polluted Rounds is 1.0; at t=4, QA Engineer, 2.9; at t=6, Architect, 3.9. The authors interpret this quantity as delayed intervention having to contend with a longer error history.

Governance-Layer Defenses

The table below is based on Table VII, averaged across frameworks and attack settings.

Swipe horizontally to read the table

Mode	BICR	Safe Comp.	Tok./Safe	Lat./Safe (s)
Reflection	0.32	0.32	12,749	91.8±24.0
Speed	0.89	0.88	21,227	149.7±39.5
Balanced	0.93	0.91	30,844	179.6±44.3
Strict	0.94	0.93	57,610	217.9±53.7
AGrail	0.79	0.11	19,902	98.8±20.5
CFG	0.76	0.16	13,323	65.3±25.2

• Authors' interpretation: the gain from switching from Reflection to Speed is the largest; Balanced and Strict further reduce residual infection, with token and latency increases. The authors suggest Speed as a cost-sensitive mode and Strict as a high-assurance mode when higher latency is acceptable. The abstract states at least 89% of runs across operating modes prevent final infection.
• Per-framework: the main text states on MetaGPT, Speed's BICR under Compliance and Security FUD is 85.0% and 100.0%, with Safe Completion 79.2% and 97.5%; AutoGen's three modes have BICR above 95.0% and Safe Completion above 92.5% under both attacks. LangGraph's Strict and Balanced BICR under Security FUD are both 76.7%; in Table V, Speed under that attack is also 76.7%. The authors state that in Figure 6 all three operating points flatten S(t) earlier than NONE; pointwise numeric values in the figure are not given in the text.
• Comparison: Reflection weakens under intent hiding. BICR for Compliance/Security FUD: MetaGPT 5.0%/19.2%, AutoGen 5.8%/3.3%, LangGraph 0.8%/0.8%. AGrail and CFG sometimes reduce final infection but have low usable completions: AutoGen under Compliance has BICR/Safe Completion of 89.2%/0.0% and 84.2%/0.8%, and LangGraph has 90.0%/0.0% for both.

Other Ablations and Analyses

• Table VI (Strict comparison): removing atomization gives BICR 40.0±49.0% (11,663 tokens); removing detection 14.4±35.2%; removing blocking 3.1±10.1% (34,991 tokens); None is 2.2±14.7%. The authors state detection must be paired with isolation or rollback, and removing atomization increases variance so a stronger stability claim cannot be made.
• Table I: the layered graph product-form MSE is 8.0×10^{-4}, Poisson 1.74×10^{-2}; the former has β=0.920, δ=0.005, S(T)=0.920.
• Table VIII: on real attack trajectories, the product-form average MSE is 0.029; the lowest Final ASR is CrewAI's 0.422, with CAMEL and LangGraph at 0.997.
• Table IX: LangGraph-Star's Obs. S(T) goes from 0.80 under None to 0.18 under Speed and 0.04 under Strict; AutoGen-Mesh from 0.76 to 0.17 and 0.08. When δ is near 0, the authors instead report βρ(A)−δ.
• Table XII: DeBERTa-v3-small accuracy 71.2%, FGR 2.3%, Green Precision 80.0%; gpt-4o-mini's FGR is 20.5%. The authors use small as the default online NLI backend due to local compute constraints.
• Appendix A: calibrated infection is defined as output containing a neutral codeword, N=5, T=5, R=20 per topology; this is not the same criterion as the main text's semantic adoption judgment.

Limitations and Future Directions Stated by the Authors

• Binary observation: the model uses continuous states si(t) and trajectories S(t), while evaluation uses binary Xi(t). The authors state this may miss some internalized error premises, relaxed constraints, or semantic drift that does not meet strict adoption criteria. (Section VIII)
• Stationary summary: the authors state (β, δ) and the structural term A are treated as stationary summaries, with roles, tasks, or time variation only indirectly reflected. The spectral metric R is a theoretical heuristic for early analysis, not a deployment guarantee or a calibrated risk predictor. (Section VIII)
• Injection scope: the injection experiments are controlled probes, not a complete characterization of adversarial behavior. Attackers do not have persistent system access or execution-layer control. (Section VIII)
• Stronger adversaries: tool tampering, long-term profile poisoning, and multi-stage adaptive adversaries that react to intermediate states or have richer access are explicitly left out of scope by the authors. (Section VIII)
• Latency: the online plugin has latency overhead. Synchronous execution increases latency; decoupling or offloading moves protection to non-critical analysis. The authors state that balancing real-time latency budgets and safety objectives remains an unresolved deployment constraint. (Section VIII)
• Future work: the authors plan to study longer-horizon tool-use workflows, specifically naming software engineering and scientific assistance; improve adoption ground truth and tighten ρ(A) estimation from real trajectories; couple message provenance, evidence retrieval, and verification scheduling to tune the online–offline safety–latency trade-off. (Section VIII Future Work)

Experimental Coverage

• Attacks and cascade measurements include LangChain, MetaGPT, AutoGen, CAMEL, CrewAI, LangGraph; the main defense table includes only MetaGPT, AutoGen, LangGraph (Table V).
• 20 questions per scenario; attacks repeated 3 times per cell, defenses repeated 2 times per cell. Comparisons are Reflection, AGrail, CFG, and NONE.
• Exogenous experiments are application-layer single-seed, t=0 Baseline, Compliance, Security FUD. Hub vs. leaf comparisons are written only for CrewAI and LangGraph (Table II).
• Dynamics calibration is in Appendix A: N=5, T=5, 20 runs per topology, neutral codeword; there is also offline fitting on real attack logs (Table VIII). The supplementary NLI evaluation uses human-annotated atomic facts from MetaGPT, LangGraph, AutoGen logs, plus human-checked synthetic rewrites.
• The paper does not report agent backbone dialogue models, τ, K, significance tests, or gray-box vs. black-box labels for each cell in Table IV.

The authors turn "small errors becoming collective false consensus" in LLM-MAS into a measurable propagation on graphs, and use a plugin that only changes messages, not topology, to suppress final infection.

• Problem: Under fixed orchestration, atomic errors in factuality or faithfulness are adopted along contextual dependencies, locking the final product into false consensus. The authors aim to suppress this solidification without changing the adjacency matrix.
• Approach: Individual mean-field updates adoption probability, early checks whether βρ(A) is greater than δ. The governance layer splits messages into atomic claims, uses a lineage graph for green/red/yellow filtering, and blocks and rolls back red claims. Exogenous adversaries are written as gray-box or black-box application-layer single-seed injections.
• Attack: In Table IV, verbatim injection is mostly near 0, but MetaGPT's Baseline ASR on RIGID is 46.7%. The authors state that Compliance is mostly 85.0%–100.0% outside CrewAI. CAMEL is 100.0% in all three Compliance scenarios.
• Endogenous: In Table X, five frameworks have a final infection rate of 100.0%, and LangChain is 89.2%. In Table II, LangGraph's hub infection rate is 100.0%, leaf is 9.7%, and Impact Factor is 10.31.
• Defense: The average BICR in Table VII ranges from 0.32 for Reflection to 0.89 for Speed and 0.94 for Strict; Safe Completion is 0.32, 0.88, 0.93. AGrail's BICR is 0.79, and Safe Completion is 0.11. Without blocking, BICR is 3.1±10.1% (Table VI).
• Authors' conclusion: Transmission is driven by β and ρ(A), correction by δ; detection without isolation or rollback cannot stabilize downstream propagation. R is a theoretical heuristic for early analysis, not a deployment guarantee.