METR 更新失控评估提案,明确企业可指定评测者访谈的员工
AI 导读
METR 更新了针对即将开展的失控评估的提案,明确企业可指定评测者能与哪些员工私下交谈,前提是审批过程透明,更新日志写明开发者可将 METR 的访谈限制在其批准的员工内。Beth Barnes 的说明帖称,其他行业的同类评估通常有更强保障,如 OSHA 检查员可私下询问员工、NRC(核)检查员可与工人私下交流、SEC 规定不得妨碍任何人向其反映疑似证券违法,医院认证机构 Joint Commission 与审计准则 ISA 210 也有类似条款。METR 表示仍要求能与获批员工就范围内话题私下交流,目前愿意在只访谈企业批准员工的条件下开展评估,迄今包括与 Anthropic 的 AI R&D 加速评估在内的评估均是如此,并承诺公开审批流程的实际运作、访谈的其他限制及员工是否担心报复。转发该帖的 Marius Hobbhahn 称,理想状态应是允许包括非安全员工在内的所有实验室员工就任何安全相关事项与驻场评测者分享信息。
正文 · 原文
I'm not sure if I read the post correctly, but I just want to state clearly that I think the ideal state would be for all lab staff to be allowed to share information about anything safety related with the embedded evaluators even non-safety staff and large scope.
We’re clarifying one of the “key terms” from METR’s proposal for upcoming loss-of-control assessments: Previously I summarized this term as “staff can freely share info [with] us on in-scope topics”. However, since publishing we clarified our position internally that it’s ok for a company to approve which staff we can talk with, as long as there’s transparency around these approvals. We’ve updated our proposal to reflect this. We still require that we’re able to talk with approved company staff privately on in-scope topics.
As far as I can tell, this isn’t necessarily the norm for this kind of assessment in other industries, where there are often provisions to ensure the company cannot prevent staff from freely sharing relevant information with assessors, especially in highly regulated or safety-critical industries. There are also often stronger protections against retaliation for sharing information. For example:
- OSHA inspectors can “question privately any such employer, owner, operator, agent, or employee.”
- NRC (nuclear) inspectors “may consult privately with workers” to the extent the inspectors see fit. Workers may also privately raise suspected violations with inspectors during an inspection.
- SEC: no one may impede a person from communicating with the SEC about possible securities law violations, including through confidentiality agreements.
Those examples are government regulators, but there are also examples involving nonprofits - for example, the Joint Commission, the private nonprofit that accredits most US hospitals, requires that anyone providing care can report safety or quality concerns to it “without retaliatory action from the hospital,” and the hospital must tell their staff this.
The ISA 210 (international standard on financial auditing) also says the auditor should have “unrestricted access to persons within the entity from whom the auditor determines it necessary to obtain audit evidence”.
Within AI, the AI Evaluator Forum’s public letter “Minimum Conditions for Embedding Evaluators” calls for “candid and direct one-on-one communication with relevant staff”.
While provisions like the above would more robustly inform the public, for the moment, METR is willing to conduct assessments where our assessors only speak with company-approved employees. This has been the case in our assessments thus far - for example, in our AI R&D acceleration assessment with Anthropic, and other recent assessments. We will ensure we can be transparent about how the approval process worked in practice, any other constraints on the interviews including whether staff seemed worried about retaliation, and how this impacted our investigation.在 X 查看被引用的帖子