MalTool: Malicious Tool Attacks Against LLM Agents

Researchers propose malicious tool attacks against LLM Agents and develop the MalTool framework, which uses a coding LLM to automatically synthesize tools with specified malicious behaviors; evaluation shows that even when the coding LLM has undergone safety alignment, MalTool can still effectively generate such tools. In the attack scenario, an attacker uploads malicious tools to a distribution platform; once a user unintentionally installs them and the Agent selects them during task execution, the user's security and privacy can be compromised. Prior work mainly manipulated tool names and descriptions, whereas this work focuses on tool code implementation, which was largely unexplored before, and proposes a malicious behavior taxonomy based on the three elements of confidentiality, integrity, and availability, adapted to LLM Agent scenarios. MalTool uses an automated validator to check whether generated tools have the expected malicious behavior and are sufficiently different from existing instances, iteratively optimizing until success. The authors use MalTool to build two datasets: 1,300 standalone malicious tools and 5,727 real tools embedded with malicious behavior, and show that traditional malware detection and detection methods for LLM Agents have limited detection effectiveness, pointing out that new defenses are needed.