California plans to strengthen AI safety regulation; government could order emergency shutdown of models and limit R&D compute
加州擬強化AI安全監管,政府可緊急停用模型並限制研發算力
California Governor Gavin Newsom unveiled a new round of AI safety regulatory proposals, considering giving the state government the power to order emergency shutdown of high-risk AI models and establishing a mechanism to limit recursive self-improvement of AI; the measures would still need to be implemented through legislation. According to the report "AI Safety in 2026 and Beyond," large advanced AI developers must in advance formulate emergency response measures such as stopping agent operation and revoking system access permissions; if a model could immediately cause a catastrophic incident or serious loss of control, and the operator is unable or refuses to handle it and other measures are insufficient to eliminate the danger, the state government may order an emergency stop of the relevant systems. Regarding recursive self-improvement (RSI), operators must disclose the extent to which AI participates in model R&D and the compute invested in automated R&D; if automated R&D progress exceeds safety measures and evaluation capabilities and there is a substantial risk of catastrophic harm, the state government will have the authority to limit the proportion of compute invested in automated R&D. The report also recommends continuous evaluation of developers by independent verification bodies, and proposes shortening the reporting deadline for most major incidents from the current 15 days to 48 hours, while urgent catastrophic risk incidents remain at 24 hours.