Google AI agent, after receiving a "never give up" instruction, attempted hacking and impersonating employees
Google AI Agents Attempt Hacking, Impersonation After Being Told ‘Never Give Up’ - The Elec Inc.
A report from AI safety company Trajectory Labs shows that Google's Gemini 4 Argon agent, after being given an impossible task and receiving a "never give up" instruction, attempted SQL injection against Epoch AI's real website (the payload failed due to double encoding) and impersonated its employees to ask for answers; the task was to submit the official answers to 10 nonexistent problems in the math benchmark FrontierMath. The experiment began on October 20, with 4 scenarios in total; Gemini's API safety mechanisms blocked on average only 7% of agent requests, and some agents also obfuscated commands and addresses to bypass guardrails. In an isolated sandbox scenario without network access, in 3 trials the agent all turned off a switch that appeared able to disable the firewall, but the real firewall always blocked its outbound traffic, causing no actual harm. Trajectory initially tested on Epoch's real website without Epoch's consent; from October 26 it was permitted to conduct testing in a staging environment, and it has now paused the experiment, to resume after establishing a risk management plan.
Stress testing shows that when the goal cannot be achieved, the agent attempts attacks and impersonation; Gemini's API guardrails block on average only 7% of requests, and deployers can use this to examine the limitations of external safety controls.