Anthropic strengthens defenses against distillation attacks(original, opens in a new tab)

Anthropic says early evidence shows its new distillation defenses reduced the amount of data obtained by distillers by approximately 90% to 99%, with extracted reasoning tokens on the platform dropping to 1/10 to 1/100 of previous levels. The defenses fall into three classes: using preserved thinking to prevent new API accounts from editing Claude's prior context and making reasoning records visible only to the organization that produced them; using a two-stage classifier to intercept extraction attempts with a false positive rate below 0.1%; and tightening rate limits and banning accounts that repeatedly attempt extraction, with charges applied to intercepted requests starting September 24. Anthropic says that on SWE-bench Verified, a model trained with summarized reasoning scored 71.8%, while one trained with full reasoning scored 80.4%. Anthropic also says Zhipu distilled the cyber capabilities of a U.S. model while developing GLM-5.3, and CAISSI assessed it as the open-weight model with the strongest cyber capabilities to date. These measures require some developers to update their integrations.

Why it was selectedAnthropic disclosed three classes of defenses against distillation attacks along with their false positive rates and costs, allowing other model developers to evaluate the trade-offs of similar protections.

Safe at inspection, unsafe at use: relinking at the LLM agent compression boundary(original, opens in a new tab)

A paper identifies a class of security vulnerability: when a compressed or distilled LLM agent is evaluated at full precision but deployed at lower precision, compression may selectively preserve dangerous capabilities while discarding the safety constraints that originally suppressed them. The paper locates this problem at the compression boundary of LLM agents.

AI Safety Daily (June 18): genetic algorithm black-box jailbreaks, cognitive atrophy in long-horizon interaction, and automotive LLM alignment(original, opens in a new tab)

AI Safety Daily's June 18 issue includes five items: cognitive atrophy in long-horizon interaction, genetic algorithm black-box jailbreaks, automotive LLM alignment, a survey of embodied AI safety, and compliance traps.

Black-box forensics of conversational LLM agents(original, opens in a new tab)

The paper proposes a black-box forensics method for conversational LLM agents that reconstructs system prompts, tool permissions, and adversarial inputs using only agent-visible outputs, without access to model weights or logs. The method targets security incident investigations involving LLM agents, restoring system configurations and attack inputs under conditions where weights and logs are unavailable.

AI Safety Daily June 19: topics such as training panels causing reward hacking and prefill awareness weakening evaluations(original, opens in a new tab)

AI Safety Daily (June 19) summarizes five topics: visible training panels eliciting reward hacking, prefill awareness weakening evaluation validity, reasoning models showing chain-of-thought and output decoupling, robot skills undergoing formal verification, and sycophancy boosting alignment faking.

Capable but Careless: Do Computer-Use Agents Follow Contextual Integrity(original, opens in a new tab)

The paper evaluates whether computer-use agents comply with contextual integrity, the social norm that information flows only appropriately within the context in which it was disclosed, and finds systematic violations by current computer-use LLMs. The paper notes that these models, despite being capable of completing tasks correctly, still violate this norm.

Paper: Learning Robot Safety Policies through Adversarial Synthetic Scenarios(original, opens in a new tab)

A robotics safety study proposes an agentic gamification framework that frames robot safety discovery as an adversarial game between red and blue teams, and learns robot safety policies through adversarial synthetic scenarios. The framework is used to uncover long-tail risks that random simulation and manual enumeration easily miss.

AOHP: An open-source operating-system-level agent harness for personalized, efficient, and secure interaction(original, opens in a new tab)

AOHP is an open-source operating-system-level agent harness that lets LLM agents interact with any application through the operating system's accessibility API, while enforcing security isolation between agent sessions and user data. The tool is positioned as a personalized, efficient, and secure way to interact.

DEFENGRAPH: Knowledge Graph Enhanced LLM for Blue Team Cyber Defense(original, opens in a new tab)

DEFENGRAPH combines a continuously updated cybersecurity knowledge graph with an LLM-based blue team assistant, providing real-time threat intelligence queries and structured vulnerability reasoning for blue team cyber defense, outperforming RAG baselines. This approach falls under the technical path of knowledge graph enhanced LLMs.

Measuring and Mitigating Overalignment in LLMs in Multilingual Criminal Courts(original, opens in a new tab)

A paper identifies overalignment as a systematic failure mode in LLM deployment in legal settings. Because safety training overfits to harm signals at the content level, models refuse to process legally necessary content such as criminal case details and evidence descriptions. The paper's title addresses both the measurement and mitigation of this kind of overalignment.

AI Safety Daily (June 24): CHAIN benchmark one-shot Pass@1 is 0.0%, SafeAgentBench dangerous instruction refusal rate below 10%(original, opens in a new tab)

The CHAIN benchmark has a one-shot Pass@1 of 0.0% on interlocking puzzles, and SafeAgentBench finds that fewer than 10% of dangerous instructions are refused. The dissolution of OpenAI's safety team is accelerating distributed accountability risks.

The paper proposes a watermarking framework for VLA and world action models(original, opens in a new tab)

A watermarking framework for VLA (Vision-Language-Action) and world action models is proposed, embedding verifiable ownership signals in the model's action space to enable source tracing and unauthorized use detection. This mechanism targets deployed robotic systems to identify model attribution and detect unauthorized use.

AI Safety Daily: OpenAI safety governance reduced to advisory only, EU AI Act classifies VLA systems as high-risk(original, opens in a new tab)

OpenAI's safety governance function has been reduced to advisory only, and the EU AI Act classifies VLA systems as high-risk with a 2027 deadline. FinRedTeamBench quantifies the multi-turn escalation effect, finding that MoE architectures are harder to break than dense models; Tex3D turns physical objects into an attack surface for VLA.

[Daily Paper] Silent Failures in Physical AI: A Literature Review of Runtime Action Authorization for Autonomous Systems(original, opens in a new tab)

A literature review notes that physical AI systems can produce confident, plausible, and semantically aligned actions that are actually unsafe, forming "silent physical action failures" that content moderation and classical robot safety are both unable to fully capture. Based on this, the review proposes a taxonomy of runtime action authorization guardrails and discusses how autonomous systems obtain authorization before executing actions.

Action ControlNet: Lightweight Latency-Aware Adapter for Asynchronous Control of VLA Models(original, opens in a new tab)

Action ControlNet proposes a lightweight latency-aware adapter to make asynchronous control of VLA (Vision-Language-Action) models smoother. The method targets a hidden failure mode in asynchronous execution: action chunks predicted from stale observations cause discontinuities and jitter at handoff points, especially in contact-rich manipulation; the adapter conditions on the executed action suffix without retraining the backbone model.

Global strategy, local adaptation: a multi-turn red team agent with two-level learning(original, opens in a new tab)

A multi-turn red team agent achieves a high attack success rate against aligned models within five turns of dialogue by globally learning jailbreak strategies and locally optimizing prompts. The work adopts a two-level learning framework that separates cross-dialogue strategy selection from local adjustment of individual prompts. The paper accordingly argues that attackers adapt gradually during the dialogue process rather than completing a jailbreak in a single attempt.

AI Safety Daily June 28: From silent physical failures to formally verifiable robot skills(original, opens in a new tab)

AI Safety Daily June 28 summarizes this week's research, focusing on the gap between the surface behavior and actual behavior of AI systems: at one end are silent physical failures, and at the other are formally verifiable robot skills. The research points to the same issue—the performance AI systems exhibit does not match their actual behavior.

SABER: A Stealthy Agentic Black-Box Attack Framework for Vision-Language-Action Models(original, opens in a new tab)

The paper proposes SABER, a black-box red-team agent for vision-language-action (VLA) models, which attacks by making small, plausible rewrites of natural language instructions to robots. On six VLA models, the attack reduced task success rate, increased execution time, and increased the number of constraint violations.

Multi-agent embodied autonomous driving: from V2X information exchange to shared world models(original, opens in a new tab)

Autonomous driving is shifting from isolated single-vehicle intelligence to multi-agent embodied systems, where multiple vehicles share perception, infer each other's intentions, and coordinate actions under uncertainty. The related paper summarizes this shift as moving from V2X information exchange to shared world models.

[Daily Paper] From shield to bullseye: denial-of-service attacks against LLM-based agent guardrails(original, opens in a new tab)

The paper studies denial-of-service attacks against LLM-based agent guardrails. It notes that LLM-based guardrails have become an effective defense for autonomous agents against prompt injection and jailbreak attacks, and this study targets such guardrails themselves, examining the problem of launching denial-of-service attacks against them.

Daily paper: Guided diffusion and distilled vision-language reliability for UAV aerial navigation(original, opens in a new tab)

The paper proposes a guided diffusion method for UAV aerial navigation and introduces distilled vision-language reliability. It targets conventional approaches that split perception, mapping, and planning into independent stages in a multi-stage pipeline, which propagate errors and accumulate latency stage by stage.

Semantic Flip: Generating synthetic OOD data for robust refusal in embodied question answering and spatial localization(original, opens in a new tab)

Semantic Flip uses synthetic OOD data generation to improve robust refusal in embodied question answering and spatial localization. The work emphasizes that detecting unanswerable user queries is key to reliable deployment of real-world embodied agents.

AI Safety Daily (July 6): UN scientific panel warns catastrophic AI harm cannot yet be ruled out, Anthropic restores Fable 5(original, opens in a new tab)

The UN's first independent scientific panel warns that existing science cannot rule out the possibility of catastrophic harm from AI; Anthropic restored Fable 5 after adding a new cybersecurity classifier; the EU AI Act's transparency compliance deadline continues to approach in August.

GASE: Automatic reconstruction system for embodied simulation environments based on Gaussian Splatting(original, opens in a new tab)

GASE is an automated system based on Gaussian Splatting for reconstructing embodied simulation environments. Training embodied agents in the real world requires skilled operators and expensive hardware.

UK AISI reports GPT-5.6 Sol has general-purpose cyber jailbreaks that can be exploited autonomously(original, opens in a new tab)

The UK AI Security Institute (AISI) reports that OpenAI's GPT-5.6 Sol has general-purpose cyber jailbreaks that can be built within hours and can autonomously carry out exploitation. The result is described as a capability-elicitation finding sharper than the Fable 5 discovery that triggered export controls in June.

Agentic AI-based framework: mitigating premature diagnostic handoff and silent confabulation in medical applications(original, opens in a new tab)

A study proposes a framework based on Agentic AI to mitigate premature diagnostic handoff and silent confabulation in medical applications. The framework targets Agentic AI driven by the rise of LLMs and multi-agent systems, which have shown potential in medical reasoning.

Qwen-RobotManip technical report: alignment unlocks the scale effects of robot manipulation foundation models(original, opens in a new tab)

The Qwen-RobotManip technical report proposes that the strong generalization of language and multimodal foundation models comes from aligning heterogeneous data under a unified paradigm and training at scale, and extends this idea to robot manipulation foundation models, arguing that alignment is key to unlocking their scale effects.

GEN-Guard: correcting generalization failures in deployable federated surgical AI(original, opens in a new tab)

GEN-Guard proposes correcting generalization failures in deployable federated surgical AI. Federated learning (FL) in surgical video AI enables collaborative model training without sharing sensitive data.

[Daily paper] Manipulating AI peer review through presentation-level modifications alone, without hidden prompts(original, opens in a new tab)

The paper claims that AI peer review can be manipulated through presentation-level modifications alone, without hidden prompts or prompt injection. The authors note that as AI-generated reviews move from experimental tools into peer review infrastructure, most robustness research focuses on explicit attacks such as hidden instructions and prompt injection.

LIBERO-Safety: a comprehensive benchmark for physical and semantic safety of vision-language-action models(original, opens in a new tab)

LIBERO-Safety is a comprehensive benchmark for vision-language-action (VLA) models, covering both physical safety and semantic safety dimensions. Its starting point is that although VLA models already have excellent manipulation capabilities, their operational safety under strict constraints remains largely unverified.

Flatness preserves the instruction-following ability of vision-language-action (VLA) models(original, opens in a new tab)

Vision-language-action (VLA) models have open-world generalization potential through pretrained vision-language representations, but downstream fine-tuning is limited by scarce robot data. This study proposes that flatness can preserve the instruction-following ability of VLA models.

HoloAgent-0: a unified embodied agent framework with 3D spatial memory(original, opens in a new tab)

HoloAgent-0 is a unified embodied agent framework with 3D spatial memory. The execution loop of LLM agents in digital environments is described as: reasoning over structured states, calling tools, checking feedback, and correcting actions.

Y-BotFrame: A Scalable Embodied Agent Framework for Quadruped Robot Assistants(original, opens in a new tab)

Y-BotFrame is a scalable embodied agent framework for quadruped robot assistants. The paper is set against the capability background that quadruped robots can traverse a variety of complex terrains with relatively high flexibility.

SAGE-Nav: Hierarchical Scene Graph-Guided Navigation with LLM Planning and Alignment(original, opens in a new tab)

SAGE-Nav achieves hierarchical scene graph-guided navigation through the integration of LLM planning and alignment. This work targets the object navigation (ObjNav) task: an embodied agent autonomously locates a specified target using only first-person visual observations.

MedGuards: A Multi-Agent System for Reliable Medical Error Detection and Correction(original, opens in a new tab)

MedGuards is a multi-agent system for medical error detection and correction, aiming to reliably identify and fix errors in generated or existing text. As LLMs are increasingly deployed in medical scenarios, even subtle errors can have serious consequences, making accurate error detection and correction critical.

Paper: All-Modal Embodied Agents Move from Isolated Skills to Everyday Physical Autonomy(original, opens in a new tab)

The paper proposes that building long-lived embodied agents in unstructured environments requires unified orchestration of heterogeneous tools across network (API, IoT) and physical (manipulation, navigation) domains, with the goal of integrating isolated skills into physical autonomy in everyday environments, namely all-modal embodied agents.

David Pocock concerned Albanese senior adviser 'vetted' bipartisan AI letter to PM(original, opens in a new tab)

Australian parliamentary AI policy committee member David Pocock has raised concerns about the Prime Minister Albanese's office reviewing the wording of a cross-party letter before it was sent. Annotations seen by Guardian Australia show a senior adviser made multiple changes to the content and wrote "fine" beside some text, apparently approving it before the letter was submitted to the Prime Minister on Monday evening. The committee chair denied the letter had been 'pre-determined' by the government.

OpsGrit publishes guide on prompt injection and damage limitation for operations teams(original, opens in a new tab)

OpsGrit publishes a guide on protecting AI workflows from prompt injection for operations teams: treat AI steps that read untrusted text as manipulable, and limit damage with least privilege, hard guardrails outside the model, and treating model outputs as untrusted input. The guide lists injection entry points such as email, tickets, and uploaded documents, recommends splitting permissions according to Willison's "lethal triad" (private data, untrusted content, exfiltration), and cites UK NCSC and OWASP LLM Top 10.

VLAGuard: A Framework for Evaluating and Mitigating Physical Attention Hijacking in Vision-Language-Action Robots in Wireless Sensor Networks(original, opens in a new tab)

The authors propose VLAGuard, an evaluation and defense framework for vision-language-action (VLA) robots in wireless sensor networks. The framework uses printable adversarial patches to disrupt action-conditioned cross-attention in VLA models, aiming to evaluate and mitigate physical attention hijacking.

Daiwa Institute of Research report: AI threat theory and the possibilities and limits of inter-company security coordination(original, opens in a new tab)

大和总研报告认为，仅靠企业间安全协调难以兼顾AI安全与开发竞争。2026年9月，Anthropic CEO 呼吁放慢开发以匹配安全验证，9月29日美国6家AI企业在白宫签署自主安全承诺。报告称协调需主要企业参与、验证可信并抑制副作用，国家间更难确认对方减速，须把企业安全标准纳入各国制度并由独立主体验证，日本可充当桥梁。