Anthropic's AI progress raises new safety concerns on the eve of its IPO
AI advances trigger new safety fears on eve of market debut
In an update to its S-1 IPO filing, Anthropic disclosed that because its overall pace of progress reached a threshold, it has triggered additional safety controls under its Responsible Scaling Policy (RSP) established in 2023. Previously, a report published by METR this month estimated that the company had achieved progress in 12 months roughly equivalent to two historical years, with model improvement speed about twice that of last year, mainly because researchers used AI tools to accelerate development. This IPO is valued at about 1.5 trillion USD, raising at most 100 billion USD. On the eve of listing, an alliance led by the Center for AI Safety and including the founders of Hugging Face and xAI issued a statement accusing it of using its safety brand to attract investors, employees and the public; Marius Hobbhahn of Apollo Research said researchers now agree that the timeline for recursive self-improvement is short.
Before its IPO, Anthropic triggered its own Responsible Scaling Policy; combined with METR's progress assessment and criticism from the safety community, this allows comparison of the lab's commitments with external doubts.