@j_asminewang 主张 safety case 机制需辅以算力等 RSI 投入指标
@j_asminewang 主张 safety case 机制不应单独依赖，还需辅以算力或其他 RSI 投入指标，因风险科学尚在发展。她指出 AI 公司/安全组织迄今只发布 safety case 草图，并反驳 Dario 称算力等节奏措施更易被钻空子，认为将 80% 算力用于抗网络攻击或治愈癌症可验证且难被钻空子。OpenAI 的 Sam Altman 称已在前沿强化学习运行前制定 safety case。
1) How can we make a safety case regime paired with embedded evaluators effective?
2) The AI industry aspiring to a standard of safety cases is good; notably the term “safety case” as used by other industries implies a high level of rigor. AI cos/safety orgs have only published safety case sketches thus far.
3) I disagree with Dario’s essay where he says he thinks pacing measures like compute are more gameable measures; I think a company allocating 80% of its compute to make critical infrastructure resilient to cyber attacks or curing cancer would be easily verifiable and hard to game, as well as good for the world.
Safety case regime is *the* thing to aim for but the science of risk assessment behind them is still being developed. So we shouldn’t only rely on the safety case regime (this will take a long time to get sufficiently right); we should also use compute (or other RSI inputs) measures.
The world deserves confidence that American companies developing increasingly capable AI will act responsibly, especially as the trajectory of progress has steepened. Every frontier lab must deliver on this, and there is no reason any of us should come to work if we cannot.
We welcome a federal framework that sets consistent safety requirements for frontier AI. But we do not believe we need to wait for an anti-trust exemption or legislation to begin the work of providing this confidence. Consistent rules to manage frontier risk so that we can maximize the benefits are a good idea (and we are excited by ideas like independent auditors).
Years ago, companies like ours developed things like Responsible Scaling Policies and Preparedness Frameworks. Those were good for that moment, and focused primarily on the deployment of completed models, not what happens during their development process.
Today's shift to focusing on safe development and evaluation will need new tools. For example, at OpenAI we now formulate explicit safety cases in advance of frontier reinforcement learning runs we expect to significantly increase capability, in addition to the safety work we have long done in advance of model releases.
We hope that other companies will learn from our approaches and propose their own; we think shared standards for misalignment, monitoring, and safety will lead to better outcomes. We look forward to collaborating with our colleagues across the industry to formulate the best version of these.
When we talk about “pacing”, we do not mean “stopping”. Progress has been rapid and will continue to be. But it should be slower than it otherwise could be; interventions like safety cases and monitoring have significant costs.
Pacing will be well worth this cost; no amount of American competitive pressure should justify recklessness, or let capabilities get ahead of alignment and monitoring.
Where we will need the help of our government is for international coordination. But first we should do what we can ourselves.在 X 查看被引用的帖子