Less Safe Over Multiple Turns: A Benchmark and Defense for Multi-Turn Safety Risks of Tool-Using Agents

Researchers propose the MT-AgentRisk benchmark, which uses a taxonomy that transforms single-turn harmful tasks into multi-turn attack sequences to systematically evaluate the multi-turn safety of tool-using Agents; experiments show that under multi-turn settings, the attack success rate (ASR) of open-source and closed-source models rises by an average of 16%, and the authors say this is the first benchmark for evaluating the safety of multi-turn tool-using Agents. The paper also proposes ToolShield, a training-free defense method independent of specific tools, in which the Agent, when encountering a new tool, autonomously generates test cases, executes them and observes downstream effects, and distills safety experience for deployment use; experiments show it reduces ASR by an average of 30% in multi-turn interactions. Code is open-sourced at https://github.com/CHATS-lab/ToolShield。