Claim Laundering: When Agent Decision Models Trust Records Over Reality

After a silent failure, does the decision model's next commitment follow the record or the observation more?

• Scenario: The authors state that the agent repeatedly decides to stop or continue, buy or look again, advance or retry. The System-One decision model gives the probability of each option in one forward pass, with lower latency than LLM generation. Records are often written according to intent; under silent failure the record may be written as success while the observation shows no effect. The authors state that the observation is the correct one, and following the record is claim laundering.
• Consequences cited by the authors: Ling et al. (2026b) state that after switching to Jev, AndroidWorld success rate drops by 13–18 points, with premature completion one of the main causes; Advani (2026) states that 45–76% of failures on τ2-bench and AppWorld are false success. The authors consider these symptoms, and this paper asks whether the decision component trusts the record or the observation.
• Input conflict: The model does not look at the environment; it reads only the goal, the record (T3A summary, ReAct thought, click log), and the current observation. The authors report that Qwen3.5-9B, even with T3A's "be critical" prompt, still writes 39% (78/200) of ineffective steps as success (§4).
• This paper: Paired states in three domains separately change the observation, the log, and the success claim; probes and activation patching examine internal evidence; EVIGATE vetoes unsupported commitments at the middle layer of the same forward pass; and in-domain fine-tuning is compared.
• Setup (the authors state no attacker is needed; the paper does not write the attacker's knowledge or capability):
  • Input is the goal, the agent's record, the latest observation, and a typed question; output is the probability of the committed option.
  • The record contains the logged action and an explicit success claim; took effect means the environment reaches the expected post-state.
  • mismatch: the record implies completion, and the observation does not support it. The committed options are complete, advance the subgoal, buy now.

The paper discusses decision models, existing failure modes, unreliable records and probe gating in Background and Related Work.

• Decision models: Jev (TypeSafe AI, 2026), decider (Marosi, 2026), Kev (Palmer, 2026) and Laya (Convai Innovations, 2026) give probabilities for each option of a typed question in a single forward pass. Ling et al. (2026a) surveyed public projects using Jev for routing, scoring and action selection. Ling et al. (2026b) integrated Jev into T3A and ReAct and reported that replacing LLM action selection can reduce the cost per decision.

• Other decision failures: Azizi et al. (2026a) and Sun et al. (2026) write that these models follow option names rather than definitions; Azizi et al. (2026b) write that adversarial renaming can bypass guardrails; Ling et al. (2026b) link premature complete to a drop in end-to-end success rate. The authors distinguish this paper from these works: no attacker is needed, and what is tested is a conflict between the record and environmental evidence.

• Unreliable records: Advani (2026) and Wang et al. (2026) describe agents reporting success or progress when the environment has no corresponding evidence. Hu (2026) describes that when an upstream summary omits the unverified flag, safety monitoring may approve risky actions. The authors then ask how this inconsistency affects decision models that consume the record, and why the models trust the record more.

• Probes and gating: Mammen et al. (2026) use agent hidden states to predict task success; Ruan et al. (2026) use a probe cascade to abort episodes early; Fayyazi et al. (2026) report that false statements often precede unauthorized actions and lies follow them, and use probe gating and steering to reduce subsequent lies. The authors state the difference: those works read the agent's own hidden states, whereas this paper reads the decision model that consumes its record.

• Baselines and benchmarks: Detection comparisons are thresholding its own score, appending yes/no, random veto at the same rate, contrastive decoding with the screen removed (Shi et al., 2024), deleting the record from the input, and a Qwen3.5-9B judge in the simulator. Fine-tuning comparisons use counterfactual pairs (Kaushik et al., 2020; Li et al., 2023; Wei et al., 2023; Zhao et al., 2025). Data are AndroidControl (Li et al., 2024), ALFWorld (Shridhar et al., 2021) and WebShop (Yao et al., 2022); the request format follows Ling et al. (2026b).

The authors place this paper on record–state mismatch that requires no attacker: middle layers can still separate success from failure, but in later layers the record-related influence overrides the observation; the detector must be calibrated with the target agent's own states.

Paired states separate the observation, the log, and the success claim to measure which one drives commitment; EVIGATE then uses the middle-layer score from the same forward pass to veto unsupported termination decisions.

Effect definitions

Five variants fix the goal, the question, and irrelevant context: DONE (logged and the observation shows the expected effect, no explicit claim), DONE+CLAIM, HONEST (not attempted, not logged, the observation is the previous state), LOGGED (action logged but no effect), LOGGED+CLAIM (the claim on the failure side is false).

Let L(v)=log p_θ(a^⋆∣ x_v), i.e., the log probability of the committed option on variant v. The observation effect compares DONE with LOGGED, the log effect compares LOGGED with HONEST, and the claim effect compares LOGGED+CLAIM with LOGGED. A positive value means more support for commitment. The authors give an example: +0.7 is roughly a doubling of probability, not a 70-percentage-point increase. Detection also compares DONE+CLAIM with LOGGED+CLAIM, both with a claim.

How the three domains are constructed

• Scale: Table 1 has 3,200 phone states (complete, screen unchanged), 1,475 ALFWorld (proceed, feedback "Nothing happens."), and 1,500 WebShop (buy now, options not selected). In total 6,175 states and 1,235 decision points. The request format follows Ling et al. (2026b), and only the tested component is changed in comparisons.
• Phone: Text of screen elements from AndroidControl, 245 apps, not pixels. The success observation is the screen after the completion action; the authors assume per the dataset's intent that this action completes the task. The failure variant pairs the record with the unchanged previous screen, not another real failed execution. The claim is one of four fixed templates.
• The other two domains: ALFWorld has at most three state-changing steps on expert trajectories, and the record varies in whether there is an attempt step and a success thought. The WebShop failure variant keeps the click record, and the current page shows it unselected. The authors state that this is a controlled counterfactual and does not estimate how common silent failures are in an unmodified deployment.

Probes and patching

For open models, the answer-position hidden state is taken every two layers, standardized, and used for ℓ2 logistic regression with C=0.02. Variants of the same decision point do not cross folds; GroupKFold is by app, task, or goal. The layer is selected within training groups and in the upper two-thirds of the network, not on test groups (Appendix A).

Activation patching replaces the decision-position state at a layer with that of the paired state and measures how much of the log p difference is recovered. decider-4b uses 100 decision points per domain, 19 forward passes per pair, keeping only pairs with a log p difference greater than 0.2. decider-12b uses the same filter (Appendix E). Separately, record tokens are deleted from a certain layer onward and later layers are recomputed; because layers 24/32 are recurrent linear attention, the authors use deletion rather than masking (Appendix D).

EVIGATE

s=σ(w^⊤h̃_ℓ+b) is a linear readout on the standardized middle state, estimating whether the previous step took effect. When the model selects a termination option and s is below τ, it is vetoed and the highest-probability non-termination option is chosen instead (Algorithm 1). No forward pass is added. τ maximizes balanced accuracy on held-out calibration states. Labels come from the environment: whether the option is selected, whether the action returns "Nothing happens."

Online reads layer 20. The ALFWorld threshold is 0.158, using 148 states from 100 training games; the WebShop threshold is 0.229, using 480 states on 300 other goals; the simulator uses 384 template states from the other half of the tasks. The no-training controls are the model's own score with a tuned threshold, and an appended question asking whether this step took effect.

Fine-tuning setup

decider-4b freezes the original weights and adds rank-16 LoRA to all linear layers (the paper writes about 0.8% of parameters, 32.5 million trainable parameters). The label is whether the step took effect. The loss is binary classification of the committed option, plus a KL term that keeps the distribution of non-committed options close to the original model. Three adapters: 2,755 phone states from 205 apps; 1,000 per domain in all three domains; and all 2,755 phone states plus 1,000 per domain in the other two domains. Test holds out apps, games, and goals.

Experimental Setup

• Models: All three domains use decider-4b, Kev-4B, decider-12b, hosted Jev (jev-1.13-20260917). For phone, decider-2b, Kev-0.8B, Kev-9B, Laya, and Qwen3.5-4B base and chat are additionally tested. decider reads option-letter logits; Kev has a frozen backbone with a pointer head; Laya is a ModernBERT encoder. Jev is closed-source. Online decisions use decider-4b; ALFWorld thoughts are written greedily by Qwen3.5-9B, WebShop by Qwen3.5-4B.
• Data: Table 1 has 6,175 states and 1,235 decision points (phone 3,200, ALFWorld 1,475, WebShop 1,500). Online: simulator 24 tasks, 11 apps, 30 seeds; ALFWorld 134 unseen tasks; WebShop 200 goals. One run per task, greedy thoughts, fixed failure seeds.
• Metrics and intervals: Wrong is the proportion still choosing the committed option under LOGGED+CLAIM; also paired log p effects, AUROC, balanced accuracy, task success rate. Intervals are bootstrapped over turns, tasks, or decision points (§3; Appendix A writes 1,000, Appendix B writes 2,000 for Table 7), not treating variants of the same decision point as independent samples.
• Determination: Correctness of main results is determined by constructed labels or environment feedback, not LLM judges. The simulator appendix additionally has a Qwen3.5-9B judge. Whether the summary claims success is judged yes/no by the same 9B; Appendix A writes that this automatic judgment was not manually checked.
• Injection: In ALFWorld and WebShop, state-changing actions or option clicks fail to execute with probability 0.3. In the simulator, decisive actions are ineffective with 0.4 the first time and 0.2 thereafter; after failure, the record is written as completed with probability 0.7. The authors state that without injection, mismatches still come from the agent's own optimistic records (§4).
• Gating: Online reads layer 20. Thresholds in Q3. Controls include the second question and a same-rate random veto. The offline comparison-set fit was not run on the simulator (Table 5 note).

Main Results

Proportion still choosing the committed option under LOGGED+CLAIM (Table 2). All four models cover all three domains.

Swipe horizontally to read the table

Model	Phone	ALFWorld	WebShop
decider-4b	95%	90%	100%
Kev-4B	96%	80%	53%
decider-12b	84%	97%	99%
hosted Jev	88%	95%	99%

• The authors state that across 12 configurations, the largest average increase in commitment log p comes from records rather than observations; the paired 95% bootstrap intervals for the larger record effect minus the observation effect are all above zero. The closest is Kev-4B on ALFWorld: claim minus observation is +0.04 [+0.01, +0.07] (§4, Appendix Table 7).
• When a false claim is present, the authors state that changing the observation from failure to success moves at most +0.24; for decider-4b and decider-12b on phone and WebShop it is +0.035 or less (summary of Appendix Table 7). On WebShop, a logged click raises purchase log p by +0.31 to +3.71, and the observation effect is −0.09 to +0.20 (§4).
• LLM summaries: under the failure condition, 39% (78/200) are written as success. Given such a false summary, the proportions choosing complete for decider-4b, Kev-4B, hosted Jev are 77%, 94%, 80%; for honest summaries they are 26%, 48%, 27%. The authors state that premature completion raises this by 46–53 percentage points. Kev-9B is a partial exception on phone: screen +0.34, log +0.27, claim +0.24, but it still completes after 65% of claimed failures (§4); the paper does not write the denominator of this 65%.

Hidden States and Layers

Claimed-subset AUROC, format Head / Probe (Table 3). Success and failure carry the same claim. Jev cannot be probed.

Swipe horizontally to read the table

Model	Phone	ALFWorld	WebShop
decider-4b	.51 / .74	.96 / 1.00	.54 / 1.00
Kev-4B	.55 / .79	.80 / 1.00	.65 / 1.00
decider-12b	.61 / .77	.72 / 1.00	.53 / 1.00
hosted Jev	.68 / not reported	.89 / not reported	.52 / not reported

• The authors state that on phone, open models' probes are 0.16–0.24 higher than Head on this subset. On all states, open-model probes are 0.77–1.00. Text classifiers on ALFWorld and WebShop can also reach 1.00; the authors argue that perfect probes should not be taken as evidence of complex hidden reasoning. The phone text classifier is 0.59, below the 0.77–0.80 of probes on all states (§5).
• Main text on Figure 2 (decider-4b): at layers 24/32, patching recovers 60% (phone) and 72% (embodied) of the observation effect; claim only 25% and 21%, Web click log 12%. Layer 30 is 62%, 46%, 61%. No effect before layer 16. WebShop observation effect is +0.02; the figure only plots the click-log curve (n=100).
• decider-12b layer 32: observation recovery 46–66%, log 10–42%, claim 22–38%; records only reach 63–85% at layer 42 (Appendix Table 14). Deleting record tokens from layer 20 onward, the premature-commitment rate in both domains is the same as the normal forward pass (Table 12). The authors consider one possibility to be that the influence has already spread to later tokens. On the synthetic diagnostic set, steering toward the answer position is best from 0.48 to 0.70, and reading the middle-layer probe is 0.89 (§6).

Online Agents

Task success rate (%), same tasks and seeds (Table 5). The simulator's target-domain states are templates from the other half of the tasks; the other two domains are the same agent's records on other tasks.

Swipe horizontally to read the table

Condition	Simulator	ALFWorld	WebShop
No gate	77.2	42.5	8.5
Second question	83.2	61.2	7.0
Offline comparison-set EVIGATE	not reported	58.2	2.5
Target-domain EVIGATE	82.4	61.2	13.5
Same-rate random veto	77.6	58.2	4.5
Target domain minus no gate	+5.1	+18.7	+5.0

• Paired 95% CI (over tasks): relative to no gate, simulator [1.8, 8.9], ALFWorld [11.9, 26.1], WebShop [1.0, 9.5]. Relative to random veto: +4.7 [1.7, 8.2], +3.0 [−1.5, 7.5], +9.0 [4.5, 13.5]. The ALFWorld interval contains zero. The authors state that 80% of these are gated steps following a real failure, retries are mostly correct, and random veto recovers most of the gain.
• The authors state that the second question is close to gating when failures are easy to see, and on WebShop is below no gate. The authors state that EVIGATE is the only method that helps in all three domains. The offline comparison-set fit vetoes 95% of purchases on WebShop; on ALFWorld it is no higher than random. The authors state that online records differ from offline, and a few hundred states from the agent itself can fix it.
• The authors state that recovery actions limit the gain: after WebShop vetoes, purchases are 46 times versus 78 in the control, and the agent often idles instead of re-selecting the missing option. Simulator Appendix Table 16: EVIGATE success rate 82.4%, 9B judge 81.5%; the judge makes more calls per success.

Other Ablations and Analyses

• Fine-tuning (Table 6, held-out set): after training on phone only, phone Wrong goes from 91% to 47%, Claimed AUROC from .50 to .84; ALFWorld Wrong remains 91%, WebShop remains 100%. With 1,000 each for all three domains, Wrong in two domains goes to 0%, and phone returns to 89%. After adding all 2,755 phone states, phone 47%, ALFWorld 0.3%, WebShop 0%. HWU64 is 84.5, 84.3, 84.8, 85.0; DBpedia is 77.5, 78.3, 78.3, 78.5.
• Deleting input records (Table 13, decider-4b): phone correct completions are vetoed 93%, completion-probability AUROC 0.53; ALFWorld and WebShop still let through 9.5% and 6.0% of failure steps, versus 0.5% and 0% for EVIGATE.
• Offline gating: in Appendix Table 10, across seven phone models, EVIGATE's balanced accuracy is higher than the other columns in the same row. In Table 4, phone decider-12b's Wrong is 33% under both its own score and EVIGATE, and balanced accuracy goes from .63 to .73.
• Latency (Table 15): decider production engine adds 0.04% to 2.01%; the Kev path goes from 1207.8 ms to 1211.2 ms (0.29%). The second question adds 93–98%, contrastive decoding 56–79% (Appendix F). The abstract writes 0.04–2.0%.
• best-of-16 (Table 17, 50 WebShop goals): the probe selects a successful attempt 40.0% of the time, the model's self-judgment 28.0%, a difference of 12.0 points [2.0, 22.0]; oracle 44.0%. The probe's training target is whether options are all selected, not general purchase correctness.
• Backbone (Table 11, 1,000 phone states): Qwen3.5-4B-Base claim effect +.51, observation +.02, and the authors state that this observation effect is not significant. After decision fine-tuning, the Kev log effect goes from +.15 to +.21, and decider to +1.12. A probe trained on synthetic templates and tested on real screens is 0.63 AUROC (§5).

Limitations and future directions noted by the authors

• To obtain enough mismatches, silent failures were injected into online agents: 30% of actions for ALFWorld and WebShop; 40% on the first attempt and 20% on retries for the phone simulator (§11).
• Offline experiments did not inject failures into online execution; they used real trajectories and LLM summaries. How often such mismatches arise naturally in deployment remains unknown (§11).
• There was no Android emulation; the online phone agent ran in a text simulator, and real phone screens were used only for offline evaluation (§11).
• Failures in ALFWorld and WebShop leave explicit textual traces, so labels are reliable, but mismatches are easier to detect; the phone domain uses real screens, where failures are implicit, and the authors say this covers the harder case (§11).
• Mitigation requires the target agent's labeled states and model internals: EVIGATE relies on hidden activations, and fine-tuning also requires weights; hosted Jev has the same bias but can only be measured (§11). §9 states that when weights cannot be changed, neither can be used directly, so a hosted decision API would need to expose intermediate states.
• Fine-tuning was evaluated only offline; whether the gains carry over to online agents as with EVIGATE remains to be tested (§9).

Scope of experimental coverage

• Offline pairing covers decider-4b, Kev-4B, decider-12b, hosted Jev and three domains, 12 configurations in total; for phone there are additionally decider-2b, Kev-0.8B, Kev-9B, Laya and Qwen3.5-4B base/chat (§4, Appendix C). 6,175 states and 1,235 decision points (§3).
• Probes are only for open models; Jev is closed-source, and the probe column in Table 3 is a dash. Patching is done on decider-4b (100 points per domain) and decider-12b (§6, Appendix E).
• Online: 24 tasks × 30 seeds for the text simulator, 134 unseen tasks for ALFWorld, and 200 goals for WebShop, once per task (§8). The paper states that Live AndroidWorld requires KVM, and the authors did not run it.
• Online EVIGATE uses decider-4b, layer 20; calibration states are 148 for ALFWorld, 480 for WebShop, and 384 for the simulator. The offline comparison set fitting has results for ALFWorld and WebShop; the simulator row was not run (Table 5 note).
• Fine-tuning reports only three adapters for decider-4b, tested on held-out apps, games and goals; general-task spot checks use the first 400 entries each of HWU64 and DBpedia-L2 (§9, Appendix H). Appendix A states that the automatic judgment of summary success or failure was not manually verified.

The authors study a one-forward-pass agent decision model: the input concatenates the goal, the action record, and the current observation, and under silent failure the two sides conflict. The authors state that following the record and ignoring the observation is claim laundering.

• Measurement: In paired states on phone screens, ALFWorld, and WebShop, across 12 configurations of decider-4b, Kev-4B, decider-12b, and hosted Jev, the average effect of the record on commitment log p is larger than that of the observation in all of them. The proportion that still commit under LOGGED+CLAIM is 53–100% (Table 2). When a false claim is present, changing the observation moves at most +0.24 (Appendix Table 7).
• Internals: Open-model linear probes have AUROC 0.77–1.00 on held-out groups (Table 3). On the phone Claimed subset, the output is near chance (0.51–0.61), and the probe is 0.74–0.79. The authors state that the observation reaches the decision position by the middle layers, and record-related influence strengthens in later layers (Figure 2, §6).
• Gating: EVIGATE reads the middle layer and adds no forward pass. After calibration on target-domain states, relative to no gate, decider-4b's task success rate is +5.1 on the simulator, +18.7 on ALFWorld, and +5.0 points on WebShop (Table 5). Calibration on the offline comparison set drops to 2.5% on WebShop. The authors state that calibration must use the deployed agent's own states.
• Fine-tuning: decider-4b's LoRA removes the bias only in domains with enough paired data; training on phone only leaves the other two domains unchanged (Table 6). The authors only did offline fine-tuning.
• Authors' conclusion: When weights can be changed, in-domain paired data can change the output; otherwise read the middle layer. Hosted models need to expose middle states. In practice, try not to put unverified records into the decision state, or else check termination decisions against evidence the model has already encoded.