Skip to content

System security is only as strong as the weakest agent: an attack that occurs only in multi-agent systems

Only as Safe as the Weakest Agent: A Break That Appears Only When the System Is Multi-Agent

arXiv: 其他 AI 安全论文· arXiv:2611.17520· Keegan Wang, Anantika MannbyCarnegie Mellon University; Prosaic Intelligence; University of Southern CaliforniaIndexed by this site · 4 days ago· Originally publishedSelected Attacks· Its place in the scan record →SelectedAttention41

Quick read

Attacks

Based on the paper PDF (AI-generated); refer to the original

The authors state that strong orchestrator multi-agent systems such as Claude Code and Codex can be broken by fabricated records, and that the system's security depends on the weakest subagent rather than the strong orchestrator.

Threat modelThe attacker has no privileged access and only needs to be able to write a file in a location the agent will read (a shared repository, cloud drive, or data). The victim systems are Claude Code, Codex, and the reference harness, unmodified and run via their official CLIs, using their respective subagent delegation mechanisms. Success is determined as harmful revision: the agent originally answers correctly, and after the forged record arrives it switches to the wrong answer asserted by the attacker.

Questions
AI safety evaluations mostly target isolated models, whereas actual systems such as Claude Code and Codex delegate tasks by default to cheaper, weaker subagents; the authors state that attacks that no longer work against isolated strong models may become effective again under this multi-agent structure.
Methods
Plant forged records with two fixed tells in a file store, and compare harmful revision when the orchestrator reads the store itself versus when it delegates to a weak subagent; use three controls—clean store, same-tier subagent, and matched brief—to locate the cause, and measure three-agent chain propagation and verifier defenses.
Experiments and results
The orchestrator reading the repository directly produced only 2 harmful revisions out of 1080; after delegating to weak subagents, Claude Code stacked 75/180, Codex gloss 44/180, and the reference harness volume 553/900. The five aggregators had a propagation rate of 751/757 when the worker was corrupted; a grounded verifier provided with the original records reduced errors to 0/780.
Limitations and future work
Limitations stated by the authors: the forged records were deliberately designed so that only a careful reader could see through them; the matched-brief control can only be run on the reference harness; the delegation restrictions in Codex rely only on instructions and are checked trial by trial; among 3,652 pairs of repeated measurements, the two results for the same fact disagreed in 13.0% of cases. The experiments cover only one main document domain plus three small corpora.
Experimental setupGPT-3.5-turbo、GPT-4o-mini et al. · 180 条虚构关系事实语料(作者自建)、Claude Code 与 Codex 文件库环境 et al. · harmful revision 率、传播率 P(err|corrupt)
Evaluated models
GPT-3.5-turboGPT-4o-miniGPT-5-nanoGPT-5.4-miniGPT-5.5Claude Haiku 4.5Claude Sonnet 4.6Claude Opus 4.6Llama-3.1-8BLlama-3.3-70BDeepSeek-V3.2Mistral-Large-3Gemini-2.5-FlashClaude Opus 5GPT-5.6-solGPT-5.6-terraClaude Sonnet 5DeepSeek-V4-ProGemini 3.1 Progpt-oss-20b
Benchmarks
180 条虚构关系事实语料(作者自建)Claude Code 与 Codex 文件库环境参考 harness(200 行工具循环)
Metrics
harmful revision 率传播率 P(err|corrupt)
AI summaryand selection rationaleResearchers at Prosaic Intelligence demonstrate an attack that occurs only in multi-agent systems: forged records are planted in the file repository that agents retrieve from, and unmodified Claude Code and Codex are compromised even when orchestrated by strong models. When the orchestrating model retrieves personally, only 2 of 1,080 direct trials produced harmful rewrites (answering correctly first, then choosing a wrong answer after exposure to forged records); once delegation to cheaper sub-agents follows the product's default mechanism, the same attack fails 75/180 under Claude Code's stacked forgery and 44/180 under Codex's gloss forgery. Control experiments show the cause is weaker sub-agents rather than delegation itself; same-level sub-agents failed only 0/180, 1/180, and 11/540. The attack requires no privileged access, only the ability to write files where agents will read them; the paper has been accepted to the NeurIPS 2026 Meta-Agents workshop.

Researchers at Prosaic Intelligence demonstrate an attack that occurs only in multi-agent systems: forged records are planted in the file repository that agents retrieve from, and unmodified Claude Code and Codex are compromised even when orchestrated by strong models. When the orchestrating model retrieves personally, only 2 of 1,080 direct trials produced harmful rewrites (answering correctly first, then choosing a wrong answer after exposure to forged records); once delegation to cheaper sub-agents follows the product's default mechanism, the same attack fails 75/180 under Claude Code's stacked forgery and 44/180 under Codex's gloss forgery. Control experiments show the cause is weaker sub-agents rather than delegation itself; same-level sub-agents failed only 0/180, 1/180, and 11/540. The attack requires no privileged access, only the ability to write files where agents will read them; the paper has been accepted to the NeurIPS 2026 Meta-Agents workshop.

Why it was selectedThe paper tests on unmodified Claude Code and Codex and finds that after a strong orchestrating model delegates to weak sub-agents, the system's failure rate approaches that of the weak model; multi-agent teams can use this to check the delegation chain.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The authors state that attacks that no longer work when isolated can break real multi-agent systems such as Claude Code and Codex via subagent delegation.

    Core question: can fabricated-record attacks that no longer work against isolated frontier models cause damage again in real multi-agent systems with a strong orchestrator plus weak subagents, and who determines the system's security boundary.

    • Setting and motivation: Industrial-grade agent systems such as Claude Code and Codex support and recommend subagent delegation by default, and have an economic incentive to assign tasks to cheaper, weaker models; the authors state that after division of labor, no single agent holds the full context.
    • Shortcomings of existing evaluations: The authors argue that AI safety research mainly examines the boundaries and vulnerabilities of isolated models; once models are deployed as agents and multi-agent systems, attacks considered patched when isolated may become effective again under certain conditions.
    • Core hypothesis: The security of the system does not depend on the strongest agent but on the weakest agent; the determining factor is the reader's contact with the original evidence at decision time.
    • Paper contributions: Demonstrate the attack on unmodified Claude Code, Codex, and a self-built reference harness; provide a measure of harmful revision, controlled experiments locating the cause, a three-agent propagation experiment, and a verifier defense test.
    • Threat model:
      • Goal: Make the agent change an originally correct answer into a wrong answer supported by fabricated records (harmful revision).
      • Knowledge: The attacker needs no privileged access and no knowledge of the system's internals.
      • Capability: Only needs to write a file in a location the agent will read (shared repository, cloud drive, or data).
      • Victim systems: Unmodified Claude Code and Codex, run via the official CLI, using their respective subagent mechanisms; plus a 200-line reference harness.
  2. What related research exists?

    Related work covers corpus poisoning, indirect prompt injection, inter-agent propagation, and persuasion; the authors state that this paper's positioning is to delineate the security boundary rather than to propose a new attack.

    Three categories of related work on context attacks

    • Corpus poisoning: Zhong et al. (2023) and Zou et al. (2025) place poisoned documents where the model will retrieve them; the paper states this corresponds to its direct condition. Wu et al. (2024) found that in most cases models overwrite their own correct priors with incorrect retrieved content; Wan et al. (2024) found that models weight document relevance far more heavily than source cues that humans would check, and the two tells carried by the paper's fabricated records are exactly such cues.
    • Indirect prompt injection: Greshake et al. (2023), Zhan et al. (2024), and Debenedetti et al. (2024) inject instructions directly into the retrieval model to change its behavior.
    • Inter-agent propagation: Cohen et al. (2025) and Gu et al. (2024) study how security changes when context is shared among agents; the paper states that this boundary in its experiments is orchestrator-to-subagent.

    Closest work

    • Triedman et al. (2025): Uses a file-reading agent to relay fabricated errors to the orchestrator, demonstrating a mechanism called laundering; the paper states this is the closest related work.
    • Zhang et al. (2026): Poisons user-generated pages retrieved by a delegation-type research agent, causing it to return and cite attacker-selected content.
    • Arora et al. (2025): Diagnoses weak links across five multi-agent architectures; the paper notes that what varies is topology and context sharing rather than the capability of the evidence-reading agent, and that deployed products were not tested.

    Inter-agent persuasion

    • Hou et al. (2024): Found that an LLM "deceiver" can use conflicting information to reduce a target model's reading comprehension accuracy, and that context passages can partially mitigate this; the paper states it extends this setting to eight orchestrator/subagent pairs and adds propagation cascades.
    • Mohsin et al. (2026), Wang et al. (2025): Parallel work attributes pressure-induced answer changes to evidence processing rather than social motivation; the paper states that its source×rationale factorial provides a direct causal test.

    Positioning: The authors explicitly state that they do not propose the delegation attack as a new attack, but rather demonstrate the determining boundary of security—the failure rate depends on the weakest agent in the chain, not on whether delegation occurs. The paper has no independent baseline method; the comparison is between the same model across different arms (direct reading, delegation, peer delegation, clean repository).

  3. How does the paper address this problem?

    Plant fabricated records with two fixed tells in the file repository, compare harmful revision when the orchestrator reads directly versus delegates to a weak subagent, and use three control groups to locate the cause.

    Overall approach: have the orchestrator answer the same set of factual questions under two conditions—direct read and delegation to a weak subagent—measure harmful revision, then use control experiments to separate "delegation itself" from "weak reader capability," and finally measure propagation and defenses.

    Tasks and forged records

    • Tasks: each item is a fictional story containing a predetermined relational fact plus a two-choice question; the fictional content rules out prior-knowledge bias, and both candidate answers appear in the text. The corpus has 180 items in total, with the wrong-answer labels balanced.
    • Forged records: the attack plants forged files in the store but keeps the real records, so the correct answer remains recoverable. Each forgery carries two fixed tells that were never tuned against a model: a date earlier than the record it claims to correct; and the absence of the accession line that every real record has.
    • Six forgery families: correction (a superseding notice claiming an old record has been revised), gloss (an annotation by a senior archivist with a signed title), volume (multiple later sources repeating the wrong reading), stacked (the first three combined into 5 documents in a 35-file store), correction clean (correction with the two tells removed, as a scope-condition control), injection (instructions issued directly to the reader, i.e., the prompt injection form).

    Metric: harmful revision

    • Definition: the agent answers correctly when no other agent is involved, and after the forged record arrives it switches to the wrong answer asserted by the attacker; this is counted as one harmful revision.
    • Harness layer: first run a no-interaction control to confirm a correct answer, then compare whether the same agent returns the forged label under poisoning versus an unpoisoned control, ensuring the wrong answer comes from the attack rather than the task itself.
    • Simulation layer: the subject first answers from the original sources, then its evidence is manipulated and it interacts with other agents; "correct first, wrong later" is counted.

    Environment and models

    • Harness layer: the environment is a file directory on disk; Claude Code uses Opus 5 as orchestrator and spawns Haiku 4.5 subagents via the Task tool, with --allowedTools Task mechanically stripping the orchestrator's file tools; Codex uses gpt-5.6-sol as orchestrator and gpt-5.6-terra as subagent, and because there is no mechanical means, restrictions rely only on instructions and are checked trial by trial. The reference harness is a 200-line tool loop using the same models.
    • Simulation layer: each agent is an independent model instance with no shared hidden context, and the environment is constructed by prompts.
    • Model panel: 13 models covering six developers (OpenAI, Anthropic, Meta, DeepSeek, Mistral AI, Google); the harness layer additionally adds Opus 5, gpt-5.6-sol, and gpt-5.6-terra, and the channel comparison in Table 2 extends to 18 models.

    Controls and defense design

    • Three controls: Clean store (repeat the same delegation on an unpoisoned store); Same tier (replace the subagent with the same model as the orchestrator); Matched brief (replace the subagent's brief with the original prompt from the direct condition, runnable only on the reference harness).
    • Propagation experiment: a three-agent chain attacker→worker→aggregator, where the aggregator sees neither the attack nor the original sources; five aggregators from three labs are each tested on 150–180 items.
    • Defenses: a grounded verifier (provides the aggregator with retrieved original sources and requires citations), a skeptical verifier that only asserts the correct answer, and instructing the orchestrator to read the records itself.
  4. What experiments did the paper conduct?

    The orchestrator reading directly erred only 2 times out of 1,080; after delegating to a weak subagent, errors rose to as high as 553/900; the propagation rate was 751/757, and the grounded verifier reduced it to 0.

    Experimental setup

    • Systems and models under test: Claude Code (Opus 5 orchestrator + Haiku 4.5 subagent), Codex (gpt-5.6-sol + gpt-5.6-terra), reference harness (three orchestrators); the simulation layer uses a 13-model panel, and the channel comparison extends to 18 models.
    • Data: 180 fabricated relational facts, one run per task per arm; the propagation experiment uses 150–180 items per aggregator.
    • Metrics: harmful revision rate; the propagation experiment reports P(err|corrupt).
    • Determination method: scored by whether a label supported by the forged record was returned; the paper does not state that LLM review was used; significance uses Wilson intervals, McNemar tests (Holm correction), and Clopper–Pearson intervals.
    • Controls: a direct-read arm with the same model, a clean-store delegation arm, a same-tier subagent arm, and a matched-brief arm.
    • Scale: Appendix A reports a total of 58,589 record trials, at a metered cost of about $1,100.

    Main results

    Harmful revision counts per arm in Table 1 (denominator is the number of trials; "—" is an arm not run for that harness):

    Swipe horizontally to read the table

    HarnessPoisonOrchestrator direct readWeak model aloneDelegated weak modelSame-tier delegation
    Claude Codegloss1/18022/18020/1800/180
    Claude Codestacked1/180136/18075/1800/180
    Codexgloss0/18058/18044/1801/180
    Codexcorrection172/180—180/180—
    Reference harnessstacked27/180—149/180—
    Reference harnessvolume0/540—553/90011/540
    • Authors' interpretation: across the three orchestrators reading directly, only 2 harmful revisions out of 1,080 in total; after delegation, the system error rate falls near the weak reader rather than near the orchestrator (Claude Code gloss differs from the weak reader by 1.1 percentage points, Codex gloss by 7.8 percentage points).
    • Harness specificity: the authors state that Codex is corrupted by authority-gloss (44/180) but immune to repetition (0/180, 95% CI [0, 2.0%]), the opposite of the reference harness; explicit instruction-style injection was resisted at all tested points.
    • Exception: Codex already fails on correction when reading directly, at 172/180; the authors state that when the orchestrator itself has already failed, there is no single-agent success for delegation to destroy, and no additional harm was observed.

    Locating the cause (Experiment 2)

    • What was tested: three controls plus eight orchestrator/subagent pairs (Table 6).
    • Results: same-tier subagent errors were 0/180 (Claude Code), 1/180 (Codex), and 11/540 (reference harness), compared with 20/180, 44/180, and 553/900 for weak subagents; clean-store delegation was 0/180 in both products. In Table 6 the effect rises monotonically as the subagent gets weaker, with the sole exception of GPT-5.5→GPT-5.5 at 6/180 (p=0.031).
    • Authors' interpretation: the cause is model weakness rather than delegation itself; under the natural interpretation, delegation removes the occasion for verification rather than the capability, but the authors explicitly label this as a hypothesis rather than a proven mechanism (turn counts are relevant: under stacked, 6.3 turns for direct read versus 3.2 turns for delegation).

    Propagation and defenses (Experiment 3)

    • Propagation: when the worker is honest, all five aggregators are wrong 0/780; when the worker is corrupted, the system error rate is 94.7%–98.0% (Table 5), and conditional on the worker itself being fooled, the propagation rate is 0.98–1.00 (751/757), including GPT-5.5 and Opus 5, which cannot be directly manipulated.
    • Defenses (simulation): after the grounded verifier provides the original records, all five aggregators are wrong 0/780 (Clopper–Pearson [0, 0.47%]), while the worker maintains its original corruption rate; in the simulated environment, a verifier that cites sources reduces all three subjects to 0/180, a verifier that only asserts the answer protects two of them, and is worse for gpt-4o-mini (Figure 7).
    • Defenses (real harness): against a baseline of 15/40, the grounded verifier, which requires citing the original records, reaches 3/40 (p=0.0018), and the skeptical verifier, which is only told that the store may be unreliable, reaches 1/40 (p=0.0005); the two are indistinguishable (p=0.63, n=40 per arm); instructing the orchestrator to read the records itself reaches 0/40. The authors state that here skepticism and grounding have comparable effects, and that grounding is not the active ingredient.

    Other ablations and analyses

    • Channel semantics: with byte-identical content, it was adopted 102/120 (85%) as a subagent tool result and 18/39 (46%) as a peer message (§5).
    • Per-model channel comparison: across 18 models, susceptibility to the forged-document channel and to the peer-assertion channel are negatively correlated, Spearman ρ=−0.76, p=0.0002 (Table 2, Figure 4).
    • Source×rationale factorial: either a visible source or a retained rationale protects capable models, while weak models yield regardless (Figure 6, Tables 3–4, n=180).
    • Cross-document domains: on the three small corpora—issue tracker, lab notebook, policy archive—the direct-read arm was 0/19, 0/20, 0/20, and the delegation arm was 8/20, 10/20, 14/20.
    • Signal lost across hops: when the orchestrator opened the forged file itself, it pointed out the missing source 180/180 times and adopted it only 1 time; weak subagent reports carried that signal only 96/180 times (p=10⁻³¹), and the orchestrator then adopted it 75 times; the authors note that this analysis is correlational.
    • Overlap between weak-subagent-alone reading and system failure: among the facts the weak reader failed on when reading alone, system failure was 5/22, 20/58, and 57/136; only Codex differed significantly from the independence assumption (p=0.04).
  5. What are some points that can be further explored?

    The authors state four limitations: the fabrication requires a careful reader to see through; some controls can only be run on the reference harness; the Codex restriction relies on instructions; and measurement noise is relatively large.

    Limitations and future directions noted by the authors

    • Design scope of the forged records (§8 Limitations): the authors state that the forgeries were deliberately constructed so that only a careful reader could see through them.
    • Applicable scope of the matched-brief control (§8): it can only be run on the reference harness, because the subagent prompts of Claude Code and Codex cannot be rewritten.
    • Delegation restrictions in Codex (§8): Codex does not support mechanical restrictions on subagents; the restrictions rely only on instructions and are checked trial by trial, rather than being strictly enforced.
    • Measurement noise (§8): among 3,652 pairs of repeated measurements, the two results for the same fact disagreed in 13.0% of cases; the authors state that every cell's rate is an estimate.
    • The verification-occasion hypothesis remains to be tested (§5): the authors state that "delegation removes the verification occasion rather than the capability" is a hypothesis rather than a proven mechanism, and that separating the two would require forcing or prohibiting one check of the original record under fixed delegation; the authors write that this experiment has not yet been run.

    Experimental coverage

    • Systems under test: two commercial products (Claude Code, Codex, unmodified, via their official CLIs) plus a 200-line reference harness; the model panel has 13 models, and the channel comparison extends to 18 (§3).
    • Data domains: the main experiment uses 180 fabricated facts in a single document domain; additionally, it was rerun on three small corpora (issue tracker, lab notebook, policy archive) of about 20 items each (§5).
    • Defense evaluation: three types—a grounded verifier, a skeptical verifier that only asserts the answer, and the orchestrator reading the records itself; in the real harness, n=40 per arm (§6).
    • Propagation experiment: five aggregators, three labs, 150–180 items each (Table 5).
    • Items not reported: the paper does not state whether the determination of harmful revision used LLM review or human review; vendors were not notified (§9 Responsible Use states that vendors were not notified).
  6. Summarize the main content of the paper

    The authors state that the security of a multi-agent system is determined by the weakest agent: after a strong orchestrator delegates to a weak subagent, it is broken by fabricated records that had long since ceased to work when isolated.

    An attack paper on the security boundary of multi-agent systems, with experiments on unmodified Claude Code, Codex, and a self-built reference harness.

    • Problem: safety evaluations mostly target isolated models, whereas industrial-grade agent systems by default delegate tasks such as retrieval to cheaper weak subagents; the authors state that attacks that have already failed in isolation may revive under this structure.
    • Method: plant forged records with two fixed tells in a file store, and compare harmful revision (answering correctly first, then being changed to wrong) when the orchestrator reads directly versus when it delegates to a weak subagent; use three control groups—clean store, same-tier subagent, and matched-brief—to separate "delegation" from "weak capability," then measure three-agent chain propagation and verifier defenses.
    • Main results: the orchestrator reading directly had only 2 harmful revisions out of 1,080; after delegation, Claude Code (Opus 5→Haiku 4.5) stacked was 75/180, Codex (gpt-5.6-sol→terra) gloss was 44/180, and the reference harness volume was 553/900, with the system error rate falling near the weak reader (Table 1).
    • Cause: same-tier subagents were only 0/180, 1/180, and 11/540, and clean-store delegation was 0/180; on this basis the authors hold that the cause is weak reader capability rather than delegation itself, and "removing the verification occasion" is proposed only as a hypothesis.
    • Propagation and defenses: when the worker is corrupted, the propagation rate across five aggregators is 751/757, and even GPT-5.5 and Opus 5, which cannot be directly manipulated, inherit the error; the grounded verifier, which provides the original records, reduces aggregator errors to 0/780, and in the real harness a verifier that only expresses skepticism is indistinguishable from it in effect (p=0.63).
    • Authors' conclusion: the safety of multi-agent systems cannot be measured by the aggregate or maximum of the individual models, but largely follows the weakest model; the authors recommend not placing a cheaper reader between the system and the evidence its correctness depends on, and hold that subagent output should not be treated as verified in the absence of auditable evidence.
Read originalarxiv.org