日本国立信息学研究所教授高仓弘树认为，CVE 数量激增确与 AI 驱动的漏洞发现有关，但近期针对 Times Car、京王等企业的攻击主因是防御不足，而非 AI 攻击能力。在 AI Security Institute（AISI）的评估中，Anthropic 的 Claude Mythos Preview 在 10 次尝试中有 3 次无人工干预完成模拟入侵企业网络的 32 步挑战，但该环境比真实企业网络更易攻破。
Oct 2Fri
Friday · 1 items
XX @florian_tramerX @florian_tramer · Attention2828
Wiz Research published its 2025 year-end review, summarizing multiple prompt injection and data exfiltration studies that surfaced in agent browsers over the year, and compiling vendors' security configurations. The review listed attacks including Johann Rehberger's zero-interaction exfiltration, Guardio Labs testing Perplexity Comet with fake stores and phishing links, Brave's disclosure of indirect prompt injection in Comet, and disclosures by Tenable, LayerX, Cato Networks, and Google researchers targeting products such as Comet, Atlas, and Operator, involving URL parameter hijacking, long-term memory poisoning, and task injection. On the defense side, vendors shifted from asking before actions to multi-layer architectures, with human-in-the-loop becoming a common guardrail; differences lay in reinforcement learning, architectural isolation, and secondary LLM review. The article offered three recommendations: isolate context, retain human confirmation, and limit agents to handling only low-risk tasks.
On December 1-2, 2025, over 300 researchers gathered in San Diego for an alignment workshop before NeurIPS to discuss how to align increasingly powerful AI systems with human values. FAR.AI's Adam Gleave noted that reasoning models and coding agents went from niche to widespread in less than 12 months, with Claude and Gemini now solving 74% of SWE-bench tasks, yet models still exhibit deception, reward hacking, and sycophancy, and generic jailbreaks can achieve a 100% attack success rate against frontier models within a week. Yoshua Bengio proposed a non-agentic "Scientist AI" research program, while Anthropic released its first destructive risk report targeting Claude Opus 4, identifying nine catastrophic harm pathways.
