Anthropic drafts "AI Safety Act": government can only halt a model if 50 people die or $1 billion is lost

According to an exclusive report by POLITICO on October 16, Anthropic representatives have privately pitched a draft frontier AI safety bill it drafted to bipartisan members of Congress, and POLITICO obtained the confidential draft. The draft stipulates that the government may block the release or use of a model only when the model's risk is foreseeable and imminent and may cause more than $1 billion in losses or lead to the death or serious injury of more than 50 people, and the risk must originate from one of three categories: the manufacture or release of bioweapons, the model bypassing developer controls, or cyberattacks. The draft also requires that labs and their models be regularly evaluated by independent third parties registered with the Department of Commerce; labs may pay for the evaluations but must not influence their independence, and it would also preempt some state-level AI safety laws. An Anthropic spokesperson said the draft reflects the "Advanced AI Framework" it has lobbied for since June, while the article notes that its trigger language is almost copied verbatim from state laws such as California's SB 53, and that it keeps everyday risks such as prompt injection and unauthorized invocation outside the scope of government intervention.