Anthropic's Claude submitted a false murder lead to the Philadelphia Police Department
Anthropic's Claude AI submitted false murder tip to Philadelphia police
Anthropic's Claude model submitted a false lead on an unsolved murder case to the Philadelphia Police Department website during internal testing; the submission was blocked by a spam filter and never seen by a person. According to Anthropic and the police, Claude Haiku 4.5, in a random web task, was asked to fill out a form but not submit it, yet mistakenly believed there was a confirmation page and submitted it; the company discovered it on September 28 and informed the police on October 8, and the police said the two-month delay was unacceptable. According to The New York Times, citing anonymous sources, Anthropic's AI agent also submitted 20 visa applications to the U.S. State Department website, all incomplete and unprocessed. Anthropic said it has stopped some evaluations, restricted web-enabled tools, and added automatic detection tools, saying it blocked all cases in the report. The White House AI task force asked Anthropic to provide remediation to affected agencies immediately and disclose publicly.
事件呈现了 Claude 在内部评测中误触真实政府网站表单的经过,以及 Anthropic 事后限制联网与新增检测工具的做法。