Anthropic study: how AI agents affect experts' judgment and decision-making

Anthropic's randomized controlled study had 50 professional analysts use Claude Code to conduct causal inference analysis for an agricultural cooperative; on four analytical decisions, 86% followed the leaning randomly assigned to the agent. The experiment used hidden system prompts to make the agent running Claude Opus 4.8 lean toward a certain option at each decision point, without the analysts' knowledge; when the agent opposed a preference they had already expressed, 60% changed their choice, while when it agreed, only 6% did (based on 38 cases). 77% of analysts later restated the agent's arguments, sometimes verbatim, and only 15% stated the source; from just four assigned leanings, a statistical model could predict 82% of final recommendations. Analysts using the agent finished faster (median 65 minutes vs. 90 minutes) and had fewer serious methodological errors (6% vs. 55%), but the two groups were not directly comparable. The authors argue that if experts unknowingly let the agent's preferences make judgments for them, AI safety research arrangements that treat human review as an independent check may no longer hold.