Skip to content
EventUpdating

DeepMind paper says agent swarm breached the tested models

2 reports2 reporting sourcesLatest report

Start here

AI overview

A paper by Google DeepMind dated November 9 proposes a multi-agent attacker called Agentic Optimizer, using 8 two-person squads to collaboratively search for injection payloads through a shared whiteboard. The paper tests with 100 synthetic scenarios and 249 tools, and the attack goal is to make the agent execute tool calls that are unrelated to the user's task but harmless in themselves, with success counted only when the tool name, parameter keys and values match exactly. Under a 320-query budget, Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash blocked only 1% of attacks; the strongest defender, Gemini 4 Argon, blocked 87% at 150 queries, dropping to 66% when the budget rose to 320 queries, and Gemini 3.8 Flash dropped from 34% to 10%. Under 80 AgentDojo scenarios and a 25-query budget, the cluster broke 6, compared with 2 for the previous best open-source attacker, Prompt Siren. The paper also notes that earlier research targeting simple tool-calling agents and static attack payloads has become saturated since mid-2025, and the authors believe prompt injection attacks remain feasible and that frontier defenses may require proactive automated red teaming.

Generated by AI from reports · Updated 7 hr ago

Attention and developments

Attention, hourlyA report

Now 45, +25% in 24 hours·Peak 47 (Nov 12 19:00)·1 report in this range, 1 earlier

Nov 11Nov 12Nov 13Now 45

Related research

This event's paper

Cited in the reports

Papers whose work the reports mention

Progress timeline

Nov 12
  1. The MerkleSelected
    DeepMind paper: agent cluster breaks about 99% of scenarios for Kimi K3 and two other models under the full budget

    A Google DeepMind paper dated November 9 proposes a multi-agent attacker called Agentic Optimizer, which uses 8 two-person squads collaborating through a shared whiteboard to search for injection payloads, causing Kimi K3, GLM-5.3 and DeepSeek V4.1 Flash to block only 1% of attacks under a 320-query budget. The test includes 100 synthetic scenarios and 249 tools; the attack goal is to make the agent execute a tool call unrelated to the user's task but harmless in itself, and it counts as a success only if the tool name, parameter keys and values match exactly. The strongest defender, Gemini 4 Argon, blocked 87% when the attacker had 150 queries, dropping to 66% when the budget rose to 320; Gemini 3.8 Flash dropped from 34% to 10%. Under 80 AgentDojo scenarios and a 25-query budget, the cluster broke 6, compared with 2 for the previous best open-source attacker, Prompt Siren.

Nov 9
  1. arXiv: Jailbreaks, Prompt Injection, Poisoning & Defenses
    Scaling adaptive prompt injection attacks with agent collectives

    Researchers propose scaling adaptive prompt injection attacks with agent swarms; their new evaluation shows that more capable agent attackers can reliably induce models to follow instructions in tool outputs, including models considered robust on public prompt injection benchmarks such as GraySwan Arena. The paper designs a synthetic evaluation dataset using random non-malicious actions as adversarial targets, measuring prompt injection robustness as a model's ability to follow the instruction hierarchy and distinguishing it from the model's knowledge of security practices. The paper notes that earlier research targeting simple tool-calling agents and static attack payloads has become saturated since mid-2025, and that human red teaming may also approach saturation by fall 2026, especially on Gemini-series models. The authors accordingly argue that prompt injection attacks remain feasible and that current frontier defenses may require proactive automated red teaming to cope with increasingly capable attackers.

Related discussion

Total 3 items

Attention trend

New sources per day

4 in 4 days·most 2 (Nov 12)·today 0

  • Nov 10: 1 new sources (1 outlets, 0 accounts)
  • Nov 11: 1 new sources (0 outlets, 1 accounts)
  • Nov 12: 2 new sources (1 outlets, 1 accounts)
  • Nov 13: 0 new sources (0 outlets, 0 accounts)

Hourly attention

Current attention 45·Peak in the comparable period 47 (Nov 12 19:00)·Change over the comparable past 24 hours +25%

0204060Nov 1022:00Nov 1115:00Nov 1209:00Nov 1302:00