日本国立信息学研究所教授高仓弘树认为，CVE 数量激增确与 AI 驱动的漏洞发现有关，但近期针对 Times Car、京王等企业的攻击主因是防御不足，而非 AI 攻击能力。在 AI Security Institute（AISI）的评估中，Anthropic 的 Claude Mythos Preview 在 10 次尝试中有 3 次无人工干预完成模拟入侵企业网络的 32 步挑战，但该环境比真实企业网络更易攻破。
Oct 2Fri
Friday · 1 items
XX @florian_tramerX @florian_tramer · Attention2828
Anthropic 本周发布 Widening the Conversation on Frontier AI 倡议，邀请宗教与哲学学者讨论 AI 价值观如何形成，并实验用“伦理提醒工具”减少 Claude 在内部评估中的失准行为。Failure-First 认为道德塑造只解决一半问题，需并行对抗韧性测试：其 OBLITERATUS 研究从 Llama-3.1-8B 第 20 层残差流消融拒答方向后，50 条提示词上的攻击成功率从 22% 升至 58%。其红队语料还覆盖 171 起具身 AI 事件、258 个模型和情绪操纵等攻击族。
One week after the GPT-5.5 Bio Bug Bounty was announced, Failure-First reviewed in its AI safety daily the reception of this public bounty in the red-teaming research community, and what it means for the F41LUR3-F1R57 research program. The review also covered quieter but still influential structural findings from the same period, including how this public bounty landed among researcher groups.
Failure-First's AI Safety Daily states that embodied AI is a blind spot in current red-teaming. Feffer et al.'s Five Axes of Divergence reveals the "security theater" in existing safety evaluations, and the RAHS score offers a concrete alternative for high-risk industries.
FLIP reverse reasoning outperforms LLM-as-judge by 79.6%. Red teaming is accused of becoming "security theater," and the wave of enterprise security lead departures continues.
Wiz Research published its 2025 year-end review, summarizing multiple prompt injection and data exfiltration studies that surfaced in agent browsers over the year, and compiling vendors' security configurations. The review listed attacks including Johann Rehberger's zero-interaction exfiltration, Guardio Labs testing Perplexity Comet with fake stores and phishing links, Brave's disclosure of indirect prompt injection in Comet, and disclosures by Tenable, LayerX, Cato Networks, and Google researchers targeting products such as Comet, Atlas, and Operator, involving URL parameter hijacking, long-term memory poisoning, and task injection. On the defense side, vendors shifted from asking before actions to multi-layer architectures, with human-in-the-loop becoming a common guardrail; differences lay in reinforcement learning, architectural isolation, and secondary LLM review. The article offered three recommendations: isolate context, retain human confirmation, and limit agents to handling only low-risk tasks.
On December 1-2, 2025, over 300 researchers gathered in San Diego for an alignment workshop before NeurIPS to discuss how to align increasingly powerful AI systems with human values. FAR.AI's Adam Gleave noted that reasoning models and coding agents went from niche to widespread in less than 12 months, with Claude and Gemini now solving 74% of SWE-bench tasks, yet models still exhibit deception, reward hacking, and sycophancy, and generic jailbreaks can achieve a 100% attack success rate against frontier models within a week. Yoshua Bengio proposed a non-agentic "Scientist AI" research program, while Anthropic released its first destructive risk report targeting Claude Opus 4, identifying nine catastrophic harm pathways.
