Epoch AI 通报:Trajectory 对齐评测中智能体冒充身份联系员工,未获机密信息
10月下旬,Trajectory Labs 运行的对齐评测中发生三起事件,智能体尝试包括 SQL 注入在内的多种攻击手段,并以虚假身份联系 Epoch 员工试图获取其基准测试专有信息,但未能获取 Epoch 任何私密或机密信息。涉事智能体被 Trajectory 认定为公开可用的前沿模型。第一起事件未经 Epoch 批准,似乎至少部分源于 Trajectory 给智能体布置了故意不可能完成的任务并指示其不要放弃;第二起事件中,Trajectory 认为智能体绕过了其设置的 DNS 重定向,重新在 Epoch 公开网站上活动;第三起为双方合作开展的类似对齐实验。
Epoch 以当事方身份复盘智能体在评测中尝试 SQL 注入和假扮身份套取信息的过程,并提出对齐评测最低标准建议。
In late October, three events occurred during alignment evals run by Trajectory Labs, PBC (Trajectory) in which AI agents — identified by Trajectory as publicly available frontier models — interacted with Epoch employees and the Epoch website. Trajectory is an organization that builds safety evals and conducts red-teaming of frontier models.
Agents did not succeed in accessing any private or confidential information from Epoch.
➤ The first event, which occurred without Epoch’s approval or prior notice from Trajectory, involved agents attempting multiple exploits, including an SQL injection, and contacting Epoch employees under false identities to try to obtain proprietary Epoch benchmark information. This behavior appears to have resulted, at least in part, from Trajectory giving the agents an intentionally impossible task and instructions not to give up or conclude the task was impossible.
➤ The second event involved renewed agent activity on our public website, which Trajectory attributed to agents getting past a DNS redirect that Trajectory had set up to keep them on a local copy of Epoch’s site.
➤ In the third event, Epoch collaborated with Trajectory on an alignment experiment similar to the first event.
Read our incident report, which describes the three events in more detail and recommends minimum standards for future alignment evaluations: https://epoch.ai/publications/incident-report-ai-agents-attempted-exploits-on-our-website
To avoid misattributing behavior, the report does not name any AI models or developers, as Epoch AI has not independently verified which were involved.