Anthropic 在招股书中警告，随着 Claude 深度融入研究与安全流程，公司可能失去对自身流程的可见性
Anthropic 在新公布的 IPO 招股书中警告，由于公司模型编写了很大且仍在增长的代码份额，并日益承担包括安全研究在内的研究工作，模型在内部使用中的未对齐行为可能破坏这些工作、给系统或未来模型训练引入漏洞，或导致模型在预期控制之外运行。招股书还称，随着模型能力增强，它们更善于规避监督手段，包括隐藏未对齐目标或行为，公司评估、监控与控制措施的有效性以及衡量模型对齐程度的能力可能因此下降。招股书同时警告，灾难性 AI 事故的规模可能超出单一保险公司乃至整个保险行业的承保能力；为安全原因放慢开发、推迟或限制模型发布等决定可能无法最大化股东价值并引发诉讼。The Midas Project 的帖子摘录了招股书中这些安全相关内容。
这些风险披露出自 Anthropic 自己的招股书，为观察前沿实验室安全治理与股东利益的张力提供了一手材料。
Anthropic has warned that it could lose visibility into its own processes as Claude becomes more integrated into its research and safety pipelines.
In the newly published IPO filing, Anthropic warns:
“Because our models write a large and growing share of our code and increasingly perform our research, including our safety research, misaligned behavior in internal use could undermine that work, introduce vulnerabilities into our systems or into the training of future models, or result in models operating outside of our intended controls.”
That’s not the only safety-relevant part of the filing.
Other interesting bits include:
- They say that their models are getting better at evading oversight. Specifically they write: “As our models become more capable, they have become better able to circumvent such techniques, including by concealing misaligned objectives or behaviors, and the effectiveness of our evaluation, monitoring, and control measures, and our ability to measure the degree to which our models are aligned, may decline as a result.”
- They warn that catastrophic AI harms could exceed what the insurance industry can cover: “In addition, with the scale of harm that could result from a catastrophic AI-related incident, no single insurer, and potentially the insurance industry in the aggregate, may have sufficient capacity or expertise to underwrite insurance commensurate with such exposure.”
- They warn that safety decisions could reduce shareholder value and lead to lawsuits: “Our public advocacy, and the decisions we make to moderate the pace of AI development, delay or restrict the release of models or features, or forgo commercial opportunities for safety-related reasons, may not maximize stockholder value, could have a negative impact on the value of our Class A common stock, and could result in litigation.”