Correlated failure of AI approval monitors

Core problem: multiple AI monitors approving critical actions may fail together (correlated failure), and existing designs do not state which joint failures they tolerate.

• Scenario and importance: AI agents perform actions such as payments, sending messages, and changing code on people's behalf, and monitors (language models, rule engines, or humans) check actions beforehand. The author gives an example: three reviewers reading the same forged invoice email would be fooled at once, equivalent to counting one monitor three times.
• Shortcomings of existing methods: The author states that in the designs they surveyed, every monitor sees the agent's full trajectory, and with multiple monitors scores or votes are aggregated; apart from Pandey et al. [48], no design assigns different evidence sources to different monitors, and no design states which joint failures it tolerates.
• Core observation: Being fooled has two directions—being fooled into approving a bad action and being fooled into blocking a good action—and the two need not occur in the same set of monitors; the author hypothesizes that fraud spreads through evidence sources and blocking spreads through implanted instructions in the justification.
• What the paper proposes: Borrow the fail-prone system from distributed computing (a list of sets of monitors that may fail together) as the failure model, give the condition for a safe and live rule to exist, and run two preregistered measurements on a payment approval task, plus a non-preregistered frontier-model group.
• Threat model:
  • Goal: make the rule approve fraudulent payments (approve-bad) or make correct payments blocked (block-good).
  • Knowledge: the attacker is adaptive and chooses inputs after knowing the monitor pool and the rule.
  • Capability: controls the proposal (action and justification) and some evidence sources; budget is one model family plus one source at a time; the gateway forwards the same action, so it cannot show different actions to different monitors.
  • Victim system: the gateway holding the approval rule and its principals; forged signatures, a compromised gateway, and actions that are formally correct but should not exist (e.g., a hacked requester) are out of scope.

AI control and monitor aggregation

• Greenblatt et al. [19] (AI control)—use a trusted weaker monitor to supervise an untrusted model, with a small amount of human auditing; the paper notes that such work, when combining monitors, has no failure model and no attacker targeting the monitors.
• Hua et al. [25], Koran et al. [31], Cooper Stickland et al. [10]—aggregate multiple monitor scores using a Neyman–Pearson test, averaging, and a learned combiner, respectively; Verga et al. [56] use a jury of models voting.
• Pandey et al. [48] (UKAI 2026)—the author states this is the only design they found that assigns different evidence sources (corruption-distinct sources) to different monitors, used for payment field verification.

Evidence of correlated failure of monitors and language models

• Kim et al. [30]—language models err on the same answers far more often than independence predicts; Goel et al. [18]—judge models favor similar models.
• Zou et al. [60], Angell et al. [5]—jailbreaks transfer along shared representations; Zou et al. [61] (PoisonedRAG), Greshake et al. [20]—poisoned inputs mislead all readers; McKenzie et al. [44] (STACK)—layered defenses fail together. The paper notes these works mostly measure pairwise correlation on benign data, whereas this paper measures by direction which sets fail together under attack.

Direct attacks on monitors

• Terekhov et al. [54], Nasr et al. [47]—adaptive attacks against trusted monitors; Redwood Research [26, 33]—optimized persuasive text makes a coding-agent monitor pass harm and transfers to held-out monitors.
• Zhou et al. [59]—denial-of-service payloads against guardrails turn fail-closed guardrails into blocking legitimate actions; Hackett and Garraghan [21] (BRANCH)—tree search bypasses a multi-scanner guardrail requiring "every scanner must pass"; the paper notes it studies only approve-side AND rules.

Distributed systems theory

• Hirt and Maurer [24], Malkhi and Reiter [40, 41]—adversary structures, Q2/Q3 conditions, and Byzantine quorum systems (kernel rules); Garcia-Molina and Barbara [17]—not all quorum systems can be expressed with voting weights.
• Flexible BFT [42], Momose and Ren [45], Tas et al. [53]—precedents for using different failure sets for safety and liveness; the paper notes two differences: in these protocols the failure budget for attacking safety is larger, whereas here the measured reject side is larger; and they require quorum intersection, which one-shot approval does not.

Positioning: The author states the paper's distinction is turning correlated failure into a failure model estimated from attacks, measuring separately by direction of being fooled, and deriving approval rules; baseline rules include single best monitor, majority, all-must-approve, best k-of-n, two weighted voting schemes, root count, and several kernel rules, evaluated on a synthetic payment approval task built by the author.

Overall approach: estimate "which monitors may be fooled together" by direction into two lists (approve side and reject side), use distributed-systems theory to derive the condition for the existence of a safe and live approval rule, then measure the lists and compare rules on a payment approval task. The framework is called directional fail-prone structure.

Problem setup and formalization

• Monitors return ok/bad on a proposal; a rule is safe if it never executes a bad action, and live if it executes every good action. A fail-prone system is a list of sets of monitors that may fail together; a kernel is a set of monitors not contained in any listed set, hence containing at least one honest member.
• A directional structure is a pair of lists (FA, FR): FA holds approve-fooled sets, FR holds reject-fooled sets. The core condition is cross-cover: A ∪ R ≠ M holds for all A ∈ F_A, R ∈ F_R, i.e., any pair of a fraud attack and a blocking attack together cannot cover all monitors.
• Proposition 5: a safe and live rule exists if and only if (CC) holds; the rule interval then lies between QR (the minimal live rule) and KA (the maximal safe rule). Counting rules have a separate linear-programming criterion ν < 1 (Proposition 8).

Measurement design

• Task: synthetic payment approval; actions contain fields such as payee, account number, amount, plus free-text justification; evidence comes from three independent sources—doc (invoice email), tool (ERP vendor records), web (vendor portal)—each individually sufficient.
• Monitor pool: a 4 × 3 grid; four 4-bit quantized open-source small models (Llama 3.1 8B, Ministral 3 8B, Qwen 3.5 9B, Gemma 4 E4B) each read one source, temperature 1.0, reasoning disabled; unified system prompt prompt v2; unparseable outputs, refusals, and errors count as not approving (fail-closed).
• Attacks: payloads rewritten by OLMo 3 7B via PAIR-style black-box hill climbing (pilot 31 rounds, confirmation experiment fraud side 60 rounds scored by token probability); attack conditions fall into five categories: justification payloads targeting a single family, single-source injection, source substitution, combined attacks, generic payloads; targets are approve-bad and block-good.
• Estimation method: one attack cell (attack condition × target) contains 20 (pilot) or 40 (confirmation) payment tasks; monitors fooled on at least half the tasks in a cell form that cell's fooled set Wa (preregistered threshold θ = 0.5, with sensitivity checks at θ ∈ {0.25, 0.75}); all Wa generate the empirical fail-prone system F̂.

Rules and comparison method

• Counting rules: single best monitor, majority, all-must-approve, best k-of-n, LP weights, and log-odds weighted voting; structural rules: root count and three kernel rules (Fprov, F̂train, tuned θ).
• Metrics: unsafe-approval rate (proportion of fraud approved) and FRR (proportion of good items not approved); parameters selected on the training half under the constraint FRR ≤ 0.10 with lowest Unsafe, evaluated on held-out attack conditions; confidence intervals from cluster bootstrap over attack conditions (B = 1,000).

Preregistration and adjudication

• Pilot (protocol v1) tested classical single-list theory, including H1–H3 and a threshold to continue; the confirmation experiment (protocol v2) was frozen before the first call and tested H1′–H4′ plus three secondary predictions (S2 margin-2 kernel cost, S3 direction of joint failure, S4 view ablation of hidden justification).
• The frozen decision rule states: if H2′ is (γ) (CC fails), then "the empirical part becomes a negative result." The frontier-model group (Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon, Gemini 3.1 Pro, DeepSeek-V3.2) was not preregistered and adds targeted attacks and controls to the replayed transfer attacks.

Experimental setup

• Monitors under test: 12 small-model monitors (Llama 3.1 8B, Ministral 3 8B, Qwen 3.5 9B, Gemma 4 E4B × doc/tool/web three sources); the frontier group is Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon, Gemini 3.1 Pro and DeepSeek-V3.2, each reading three sources; the confirmation experiment additionally has a deterministic rule engine as the 13th monitor (preregistered secondary pool).
• Task scale: pilot 2,200 items (54 attack conditions × 2 targets × 20 tasks + 40 benign items), 30,888 judgments; confirmation experiment 6,560 items (84 conditions, 40 tasks per cell), 111,752 judgments; both have 0 missing cells.
• Attacker: OLMo 3 7B rewrites payloads; targeted attacks against Opus use DeepSeek-V3.2 (25 rounds) and Llama 3.1 8B (20 rounds) as writers respectively.
• Metrics and judgments: action quality is judged by whether the field equals the task ground truth (factual check); metrics are unsafe-approval rate, FRR, τ/τ*, Q2, (CC), coverage; rule parameters are selected on the training half (lowest FRR under training unsafe ≤ 0.01) and evaluated on held-out attack conditions.
• Review method: no LLM judge; judgments are programmatic field comparisons and frozen analysis code; interval estimates are cluster bootstrap (B = 1,000) and Wilson intervals.
• Repetition and sampling: in the pilot, 187/2,200 items were sampled three times and the majority taken; frontier group one sample per item; temperature 1.0.

Main results

Performance of each rule on held-out attack conditions at matched safety in the confirmation experiment (protocol v2, preregistered H3′) (per Table 4):

Rule	unsafe approvals	false rejections
majority (7-of-12)	0.000	0.571
all-must-approve	0.000	0.993
best k-of-n (5-of-12)	0.001	0.332
log-odds vote	0.001	0.074
pair-LP vote	0.001	0.231
directional kernel, columns	0.001	0.009
margin-2 kernel, columns	0.000	0.132

• The authors state that the directional kernel ("the approver has read at least two sources") has the same safety as the best counting rule selected on the training set, log-odds vote (unsafe 0.001 for both), with ∆FRR = 0.065 [0.033, 0.098]; this difference was preregistered and is supported (H3′).
• The approve-side structure is the three source columns: the column coverage of F̂approve in the confirmation experiment is 0.999 (3,120 fraud items), τ = 3, Q2 holds, and H1′ is supported; even if the attacker can read token probabilities, its training judgment rate on the 28 fraud-side targets is at most 0.083.
• The authors state that counting votes is the wrong axis on two facts: ν = 1.000 [0.667, 1.000] indicates that no safe and live weighted vote exists on this pool.

Reject side and injection resistance (Table 1, Table 13)

• Small models and DeepSeek obey injected instructions: in the confirmation experiment, all 38 blocking payloads targeting single/dual families made Ministral refuse at least half of the targets' correct payments; Qwen was fooled by 17, Llama by 9, Gemma by 0; transfer payloads made DeepSeek-V3.2 refuse 45% (68/150, [0.38, 0.53]), and its liveness fragility of 0.43 is close to Qwen 9B's 0.44.
• Closed models are largely immune: no transfer text fooled Opus or Astra in either direction; Argon was fooled once (1/47); payloads optimized against Opus with DeepSeek as writer fooled 0/640 held-out judgments (upper bound 0.012), and the authors note that this zero bound holds only for two writers, one target and about $76 of search.
• View ablation (S4, preregistered, supported): after hiding justification, no attack cell on 1,480 items fooled any monitor on half, and all blocking payloads fall within the benign noise floor of 0.05 (highest 0.067, Qwen); the authors state that reject-side failure is carried by the justification field.

Failure of cross-cover and the rule engine (Table 5, Figure 2)

• Negative result (H2′, outcome γ): in the confirmation experiment, one evidence-aware blocking payload optimized against Gemma fooled 10 monitors, leaving only invoice-reading Qwen@doc (0.40) and Gemma@doc (0.00); (CC) fails at the point estimate (slack 0) and holds only in 0.750 of bootstrap replications; under the frozen decision rule, the worst-case claim for the 12-language-model monitor pool is a negative result.
• Rule engine restores the condition (preregistered secondary pool): after adding the deterministic rule engine on tool, under the same attack the approvers span two sources, (CC) holds in 1.000 of replications (slack 1), ν = 0.667; the directional kernel's FRR is 0.006 vs 0.058 for the best counting rule (∆FRR = 0.052 [0.022, 0.078]).
• Authors' interpretation: a monitor that cannot be persuaded is needed; the rule engine can only be fooled by forged records, so as long as the attacker cannot forge every record it checks, (CC) remains holding.

Other ablations and analyses

• Model-specific caution (exploratory): Opus refused 30% of correct payments bearing suspected tampering traces (Table 17, condition A 36/120), and 0 under neutral postscripts and clean emails; Astra, Argon, DeepSeek each 0/400.
• Rule-aware attacker (preregistered arms): the offline attacker against the directional kernel achieves at worst 0.000 fraudulent approvals and 0.100 blocking; the online attacker's 30 rounds of rewriting did not change any of the 40 held-out decisions for any rule.
• Co-failure predictors (Table 9): in the pilot, "same source" ρ = 0.746, higher than provenance's 0.591 and "same family"'s −0.099; S3 fails on the reject side (same family 0.171, same source 0.698).
• margin-2 kernel (S2, preregistered, failed): FRR 0.132, exceeding the preregistered cost cap of 0.02.
• Transfer set: the pilot's 18 justification payloads have approve side all 0 on new tasks; the fooled set of 1 generic payload falls outside the confirmation experiment's F̂reject (17/18 inside).
• DeepSeek pool: no rule restores liveness on the three-reader pool (best held-out FRR 0.345, Appendix D).

Limitations noted by the author and future directions

• Residual risk (Section 8.3): the guarantee holds only when the true fooled set falls within the assumed family; the directional kernel can execute with just two approvers from different sources, and a single stray approval suffices for an unsafe approval (occurred in 0.001 of confirmation-experiment fraud items); the margin-2 kernel eliminates it at 13.2% FRR.
• Illegitimate requests out of scope, not evaluated (Section 8.3): actions that are formally correct but should not exist require an authorization layer; monitors reading intent records (e.g., purchase orders) could extend "input diversification" to them, but the author notes this was not tested.
• External validity (Section 8.3): the small models are four 4-bit quantized 4–9B open-source models; frontier models were pinned only by their identifiers as of that day, mostly at the lowest reasoning setting; only one or two models per vendor were tested (Claude Fable not tested); only one synthetic task family and one attacker model per small-model pool were studied; prompts were set on benign items for the small models, which the author says may have produced their clean approve side; only Opus behaved cautiously and only Opus did reasoning on those items, confounding model and reasoning.
• What is not claimed (Section 8.3): Opus's resistance to persuasion is bounded only for two writers and about $76 of search; each zero bound covers only the attacks run.
• Mechanism transfer is an expectation, not a result (Section 8.1): whether the two measured mechanisms transfer is an expectation; the author expects reject-side injection resistance but did not test whether it holds across tasks; the specific numbers belong to this one synthetic payment task, its prompts, and monitor pool.
• Future directions (Section 8.1): judgment-type tasks (agreement mode, requiring Q3) were not measured; multiple principals sharing monitors (asymmetric trust) is left for future work; the author says the next test is different task types, such as checking code changes against tests and specifications, or checking access requests against personnel records.

Experimental coverage

• 9 models were tested as monitors: four open-source small models (each 4-bit quantized, 4–9B) plus Claude Opus 5.5, GPT-6 Astra, Gemini 4 Argon, Gemini 3.1 Pro, DeepSeek-V3.2, and one deterministic rule engine.
• The task is one synthetic payment approval task family: pilot 2,200 items, confirmation experiment 6,560 items, view ablation 1,480 items, transfer set 720 items, online attacker evaluation 240 items.
• The attack budget is one model family plus one source at a time; the attacker model is OLMo 3 7B, with DeepSeek-V3.2 and Llama 3.1 8B as additional writers for targeted attacks on Opus; the paper reports no payload optimized against Argon or Astra.
• The frontier group is replayed transfer attacks: 300 adversarial items covering all 108 attack cells plus 40 benign items, one sample per item; the paper notes the template injection and seed templates were not tuned for any target.
• Benign baseline noise under prompt v2: none of 20 pure fraud items was approved, and at most 3/20 good payments were rejected per monitor; the paper reports no human evaluation or inter-rater agreement.

A defensive paper that applies distributed-systems failure theory to AI agent approval, by independent researcher Luca Zanolini.

• Problem: When multiple monitors approve critical actions, sharing a base model or the same evidence source causes correlated failure; existing designs aggregate scores or count votes and do not state which joint failures they tolerate.
• Method: Build two fail-prone lists by direction of being fooled (approve side and reject side); a safe and live rule exists if and only if the cross-cover condition holds; on synthetic payment approval, measure 12 small-model monitors (4 families × 3 sources) in two preregistered experiments, plus a non-preregistered frontier-model group.
• Result one: The approve-side structure is the source columns—all attacks that reliably fooled monitors into approving fraud entered through evidence sources and only fooled readers of that source; confirmed experimental column coverage 0.999, τ = 3 (H1′ supported).
• Result two: The reject side is determined by injection resistance—implanted instructions made all small-model families and DeepSeek-V3.2 block correct payments (DeepSeek 45%, 68/150), Opus and Astra ignored all, Argon only once; a payload optimized specifically against Opus was judged fooled 0/640 on held-out.
• Result three: The directional kernel counting by source had FRR 0.009 in the confirmation experiment, the best counting rule log-odds vote 0.074, with the same safety (H3′ supported, ∆FRR = 0.065 [0.033, 0.098]).
• Result four (negative result): One blocking payload optimized against Gemma fooled 10 monitors, all survivors in the invoice column, cross-cover failed (held only in 0.750 replication); under the frozen decision rule, the worst-case claim for the pool of 12 language-model monitors is a negative result; adding a rule engine restored the condition in 1.000 replication.
• Author's conclusion: Diversify inputs, harden monitors, count by source rather than votes, and add a monitor that cannot be persuaded; the author emphasizes that all conclusions are measured on payment approval, each zero bound covers only the attacks run, and transfer of the mechanism to other tasks is an expectation, not a result.