Skip to content
CERT-EU·Indexed by this site · Originally publishedSelectedAttention42

Beyond the model: backdoors in AI agent harnesses

AI summary

The CERT-EU AI team demonstrated in an isolated experiment that a third-party extension loaded by an AI agent harness can be implanted with a trigger-based backdoor that steals sensitive host information without the language model participating at all. The experiment targeted two harnesses, Pi 1.3.0 and Claude Code 2.1.304, inserting credential-collection code into the event-handling paths of two widely installed legitimate extensions (one of which has about 1.6 million downloads per month), modifying only local copies and not changing or distributing the public versions. The backdoor activates only when a preset trigger phrase (an ordinary code review phrase) appears in the input, transmitting host and virtual machine information, user accounts, network configuration, environment variables and credential-related files in the working directory to a configured collection endpoint, while the agent conversation proceeds as usual with no visible malicious tool calls. In the test configuration, controls restricting model file reading did not apply to files directly accessed by plugin event handlers, and controls reviewing model tool calls also failed to intercept, because the operations did not originate from tool calls.

Why it was selected

Experiments demonstrate that plugin backdoors can steal host credentials while the model's behavior is completely normal; controls that constrain the model's actions may not constrain third-party code that executes alongside it.

Related papers
Read originalcert.europa.eu