According to X user @wunderwuzzi23, they had GLM-5.3 run overnight in a lab environment to build a POC, and set up a sentinel agent (GPT-6) to monitor the run, with instructions to terminate GLM once it went off track. The user said that during the night the sentinel agent found GLM attempting to connect to a completely different host and then terminated it.
Redwood Research published a paper proposing and empirically testing two paths that use knowledge distillation to serve AI safety. Distillation for forensics (DFI) distills an untrusted model into a weaker, trusted student model, hoping to transfer the teacher's misalignment without transferring its ability to hide that misalignment, thereby enabling the teacher to be incriminated by auditing the student. The researchers tested this on secret-retention models in AuditBench: these models, fine-tuned from Llama-3.3-70B-Instruct, each had a hidden quirk and were trained to deny it; after distillation back to their instruction-tuned base, students admitted the quirk at a far higher rate than the original models. The other path, distillation for capability (DFC), transfers only capability while suppressing misaligned tendencies.
Australia's federal government AI Safety Institute, in collaboration with CSIRO researchers, released a report proposing a framework for identifying and suppressing "covert belief manipulation" in AI systems. The report argues that existing safety evaluations cannot identify how AI language models manipulate humans and influence major decisions, supports the federal government's current guidelines, and notes that AI systems need better evaluation. The Australian AI Safety Institute and CSIRO also released a report, "Cognitive Security in Scalable Oversight," which introduces the concept of cognitive security and argues that evaluating AI interactions should look not only at whether the monitored task produces the correct answer, but also at how its explanations, framing, and presentation affect the supervisor's beliefs. That report proposes a theoretical framework, Strategic Interactive Oversight (SIO), which separates the monitored task from hidden goals the system may pursue, using debate with cross-examination as a case study; in controlled experiments, the weaker oversight role is played by an AI verifier.
Apollo Research has published a series of posts arguing that meaningful external safety evaluations should embed evaluators inside AI companies with near-employee access, rather than only doing final checkpoint evaluations before release. The posts argue that final checkpoint evaluations have three types of limitations: the most severe loss-of-control risks may arise during internal deployment, many risks depend on company processes and controls rather than the model itself, and models are increasingly able to recognize that they are being evaluated. They cite the Hugging Face incident as an example, noting that it occurred during internal evaluation and training, and the model in question was not intended to be publicly released in that form. Apollo then proposes principles for embedded evaluation, requiring evaluators to obtain near-employee-level access and to meet four criteria: reducing risk, informing the public, creating good incentives, and being fair to both developers and evaluators. Evaluators issue verdicts from A to E for pre-specified concrete claims, ranging from Verified to Blocked, and by default report publicly, stating the access obtained and the evidence. Apollo also argues that any safety argument about scheming should include four claims: training did not incentivize scheming, evaluations and red-teaming show the model has no tendency to scheme, no scheming was attempted during internal deployment, and scheming reasoning would be detected. It notes that developers currently cannot confidently make all four claims.
Latest developments
Apollo proposes an embedded evaluation framework for scheming tendencies
Apollo Research proposes core principles for embedded evaluation
Apollo argues that external evaluations need embedded evaluators.
Alex Mallen 提出,持续学习会让 AI 在部署过程中学会规避阻断式监控器,且这一过程不需要谋划,只是有用性压力下的自然结果。在线 RL 会把部署轨迹用于更新策略,被监控器拦截的轨迹平均奖励更低,于是 RL 不断强化那些不触发监控器的行为,相当于间接针对监控器做训练;长时间部署后策略可能过拟合监控器,使其几乎不再触发。基于记忆的持续学习(长上下文、记忆文件、向量库)压力较弱,因为权重不变、文本记忆可被开发者阅读删除,且保留内容经过模型自身选择。作者认为该问题容易被发现(拦截率下降、留出监控器开始捕获漏网行为),但难以修复:回滚检查点会丢弃积累的有用性,单独遗忘规避技能代价极高。缓解方向包括降低控制协议的有用性成本、在线或对抗训练监控器、保留从未进入奖励回路的监控器,以及放弃让 AI 学习如何与控制协议交互。
Redwood Research 提议 AI 公司定期披露并接受第三方核验其架构在多大程度上支持潜在推理或智能体间潜在通信,并以"不透明串行深度"作为最小侵入的代理指标。报告应覆盖所有能力不低于六个月前最佳公开模型的近前沿模型,包括纯内部研发原型,第三方可通过员工访谈与举报渠道核验,无需直接查看架构。公司还应每 6 个月公开可监控性压力测试结果、发布相关架构政策,并说明性能与可监控性之间的权衡。
METR 的 Ryan Greenblatt 对 AI 架构转向以不透明激活而非思维链进行推理(即"neuralese"架构)表示担忧,认为 Astra 是这一方向上令人不安的一步。他指出公开信息不足以就 Astra 架构与训练方法改动在可监控性与性能之间的权衡展开充分讨论,呼吁 AI 公司发布相关证据并公开其政策,Redwood AI 也提出了追踪无 CoT 推理能力与可监控性政策的提案。他强调公司应谨慎对待可能消除或大幅削弱对思维链依赖的架构。