Skip to content
Help Net Security AI· Anamarija Pogorelec·Indexed by this site · Originally publishedSelectedAttention70

Microsoft researchers demonstrate hidden backdoor technique targeting AI agents

Microsoft researchers demonstrate hidden backdoor technique for AI agents

AI summary

Microsoft researchers demonstrated a hidden backdoor technique called Lazarus that, by modifying fewer than 10 dormant neurons, can make an AI agent execute attacker commands when specific trigger conditions occur, without retraining the model. For example, a date change can activate the backdoor during routine tasks, prompting the agent to download and run attacker-controlled code using existing permissions; the attack requires the ability to modify model weights, which compromised accounts, development pipelines, or storage systems could all provide, and both in-house development and third-party models are affected. The modified model behaves normally before triggering, making it hard for standard safety evaluations to detect; Microsoft said it has not yet observed threat actors weaponizing the technique. Microsoft recommends comparing checkpoints against the declared base model before deployment and after each update; reliable detection requires comparison with the original unmodified weights, along with least privilege, sandboxed execution, restricting outbound traffic, and requiring human approval for high-impact tool calls.

Why it was selected

The article presents the backdoor's trigger mechanism and detection conditions; the audit approach based on weight provenance comparison can serve as a reference for teams deploying AI agents.

Read originalhelpnetsecurity.com