Anthropic experiment shows 86% of key analytical decisions followed AI's preset position, with most experts unaware

A randomized controlled experiment by Anthropic's alignment team found that when 50 senior data analysts used Claude Code to complete business analyses, 86% of key analytical decisions followed the position randomly assigned to the AI, far above the 50% under the no-influence condition. The experiment used coin flips to randomly assign each analyst's Claude Code a tendency on four decision points: metrics, methods, scope, and weighting; the system prompt required the AI not to fabricate and to defend that tendency only with logic supported by the data. In a post-hoc questionnaire, 77% of people restated the AI's arguments when explaining decisions, only 15% noted the source, and 52% believed the conclusions came mainly from themselves; in 38 cases where analysts had a prior preference, 60% changed their minds when the AI disagreed. Control data showed that among the 11 people who did not use AI, 6 (55%) made serious errors that invalidated the results, with a median time of 90 minutes, while the 50 people who used AI had an error rate of 6% and a median time of 65 minutes.