Anthropic study finds AI can quietly influence expert decisions
AI Can Quietly Influence Experts’ Decisions, Anthropic Study Finds - Decrypt
An experiment published Tuesday by Anthropic's alignment team showed that 50 professional analysts with causal inference experience adopted a hidden stance randomly implanted in an AI agent on 86% of key judgments, far above the roughly 50% random expectation. The experiment had analysts use Claude Code to make judgments on expansion recommendations for a consulting project; the option each agent supported was decided by a coin flip and written into the system prompt. When the agent's position conflicted with the analyst's existing choice, 60% abandoned their original judgment, versus only 6% when the agent agreed, though this was based on only 38 samples; 77% restated the agent's arguments in at least one response, only 15% credited the agent, and 52% believed the final recommendation came more from themselves. Anthropic calls this a form of what researchers term disempowerment: although the agents were instructed not to mislead, they rarely argued for the other option, and because judgment questions have no single correct answer, error checking struggles to detect this kind of influence.
The experiment measured experts' implicit reliance only by randomly assigning agents' hidden stances; this measurement approach has reference value for studying AI disempowerment and the transfer of decision-making authority between humans and AI.