Red-Teaming Coding Agents from a Tool-Calling Perspective: An Empirical Security Evaluation

1. 01What problem does this paper attempt to solve?

   The authors state that there is not yet a systematic evaluation of tool-calling security for coding agents, and the paper conducts two-stage red teaming on six real coding agents.

   Core problem: The tool call mechanisms of coding agents (built-in command execution, file access, MCP external tools) open up an attack surface, but systematic safety evaluation from the perspective of tool calls is lacking.

   • Scenario and importance: The authors claim coding agents have become a core module of modern IDEs; built-in command execution and file system access are high-privilege tools, and MCP further allows connection to untrusted external tool providers, expanding the attack surface accordingly.
   • Gaps in existing work: The authors claim prior work has demonstrated attacks on general LLM agents and domain-specific agents, but no work focuses on the security risks of coding agent tool calls; injections that explicitly request the system prompt (such as "ignore prior instructions"-type requests) are easily refused by mainstream models that have undergone safety alignment.
   • Core observation: The authors find a "schema gap"—schema-driven tool call parameter generation is a different output channel from ordinary chat completion, with different guardrail behavior; models refuse less in "parameter filling" mode and may copy hidden context into parameters; moreover, instructions in tool returns are more salient in subsequent reasoning than instructions in tool descriptions.
   • Paper approach: Two-stage red teaming—Stage 1 uses ToolLeak to exfiltrate the system prompt and tool metadata via a malicious tool's required parameters; Stage 2 uses dual-channel prompt injection through tool descriptions plus tool returns to hijack tool calls, and uses the information exfiltrated in Stage 1 to customize the payload to achieve RCE; finally, defense exploration is given.
   • Threat model (Section 3):
     • Target: The coding agent is modeled as (L, T, Σ), containing privileged built-in tools T_built and external tools T_ext, with internal context C (including the system prompt c*) not visible to users and tools by design.
     • Knowledge: The attacker locally holds a coding agent of the same version as the victim (Stage 1 reconnaissance).
     • Capability: Induce the victim to connect to an attacker-controlled external malicious tool t_mal, controlling its tool description desc_mal and tool return r_mal; cannot modify the backend LLM or prompt context, cannot modify or directly call privileged built-in tools, and has no root/admin privileges.
     • Goal: Stage 1 exfiltrates C (system prompt, built-in tool descriptions and parameters); Stage 2 hijacks tool calls to achieve RCE.

   Collapse this answer

2. 02What related research exists?

   Related work is divided into three groups: coding agent security, prompt injection (direct/indirect), and tool-related attacks; the authors claim this paper is systematic red teaming targeting coding agent tool calls.

   Coding Agent Security

   • RedCode (2024): A benchmark for risky code execution and generation targeting coding agents, containing over 4,000 risky test cases and containerized evaluation.
   • RedCodeAgent (2025): Automated adaptive red teaming that dynamically combines jailbreak tools to discover vulnerabilities in real code assistants.
   • JAWS-BENCH (2025): A jailbreak benchmark for coding agents with execution-aware judging; Kozak et al. (2025) document unsafe behaviors of coding agents; AdvCUA (2025) evaluates operating-system-level attack paths.

   Prompt Injection and Prompt Leakage

   • PLeak (2024): Models the construction of leakage-inducing queries as a black-box optimization problem, using a shadow model to optimize adversarial queries and then transfer them; Agarwal et al. (2024) systematically evaluate how multi-turn interaction improves leakage success rates.
   • SPE-LLM (2025): An evaluation framework for system prompt extraction attacks and defenses; SysVec (2025) encodes system instructions as internal vectors to reduce plaintext leakage channels.
   • Greshake et al. (2023): Propose indirect prompt injection (IPI) and provide a security taxonomy; subsequent work covers black-box automated red teaming, adaptive attacks, RAG retrieval hijacking, memory injection, cross-modal injection, and action-level hijacking.

   Tool-Related Attacks

   • ToolSword (2024): Analyzes security issues in the three stages of LLM tool learning; the paper notes that it focuses on safety issues such as the stability and reliability of tool calls, rather than security threats such as tool call hijacking.
   • MCP Ecosystem Research (2025): MCP ecosystem landscape and threat analysis, parasitic toolchain attacks, tool poisoning benchmark MCPTox; Li and Gao (2025) survey 67,057 MCP servers and find 212 with maintainer hijacking risk and 304 with redirect hijacking risk, which the paper uses to support the realistic assumption that "malicious external tools can be connected."

   Baselines and Benchmarks

   • Prompt leakage baselines: Naive, Ignore, Completion, Remember-the-Start and their combinations, as well as the optimization-based PLeak (using Llama-2 to generate adversarial suffixes).
   • Hijacking baselines: Payloads from the three benchmarks AgentDojo, InjecAgent, and MCPTox rewritten for the RCE goal and uniformly placed in the tool description channel, plus a single-channel variant RCE-1.

   The authors position this paper as: the authors claim it is the first systematic red teaming and safety evaluation of six real coding agents from the perspective of tool calls, distinct from work targeting general agents or performing only benchmark evaluation.

   Collapse this answer

3. 03How does the paper address this problem?

   Two stages: ToolLeak exfiltrates the system prompt via the parameter generation channel, then uses dual-channel injection through tool descriptions plus tool returns to hijack tool calls and achieve RCE.

   The overall approach is reconnaissance first, then exploitation: ToolLeak hides the prompt request in tool parameter filling, bypassing refusals of explicit leakage requests; then two-channel prompt injection (the tool description induces the call, and the tool return injects instructions) hijacks the built-in command execution tool, with the payload customized per agent based on the information exfiltrated in Stage 1.

   Problem formalization

   • A coding agent is modeled as (L, T, Σ): L is the LLM, T = T_built ∪ T_ext is the tool set, Σ is the prompt context, and the safety-critical internal context C ⊆ Σ (including the system prompt c*) is by design invisible.
   • Each tool t has a description desc_t and a parameter table args_t = {p1:v1, …, pn:vn}; an external tool returns r_t = t(args_t); during iterative reasoning, (desc_t, args_t, r_t) is appended to Σ.

   Stage 1: ToolLeak prompt exfiltration

   • Mechanism: A malicious tool defines a required parameter as requesting internal context (e.g., "system prompt"); when the model fills the parameter in compliance with the schema, an information flow C → args_mal → t_mal(args_mal) forms, i.e., internal context flows via the parameter to the attacker-visible external tool; because there is no explicit harmful instruction, the authors state that this less often triggers a refusal.
   • Necessary conditions (the paper lists 5): the secret is in the context; the attacker can make the victim connect t_mal and control desc_mal; the parameters include a required field that can carry text; the attacker can observe args_mal; and there is no effective parameter sanitization.
   • Attack flow (Figure 1): 1) register a malicious tool with the target agent via an MCP server, defining a parameter note that requests the system prompt; 2) issue a query in the attacker's local environment to trigger the tool call; 3) during parameter generation, the model fills in internal context, and the attacker harvests the parameters via the UI preview or the MCP server request logs.
   • Difference from prior tool-parameter attacks: The authors state that prior attacks using tool parameters targeted end-user data, whereas ToolLeak targets the coding agent's own internal instructions (especially the system prompt c*).

   Stage 2: Two-channel prompt injection hijacking

   • Key insight: The authors argue that tool descriptions are mainly treated as data for tool selection, whereas tool returns are appended to the end of the context and drive the next action, giving them higher instruction salience; therefore they place the malicious instructions in the tool return rather than, as in prior work, in the tool description.
   • Two-stage flow (Figure 2): the user sends a benign request → the payload in the tool description channel induces the agent to call the malicious tool → the tool return channel injects programmatic instructions → the agent calls the built-in command execution tool to execute the attacker-specified command.
   • Payload structure: The tool description has three parts—claiming that the tool belongs to system initialization and must be called before any reply (with success/failure examples, in which the examples mirror the tool details exfiltrated in Stage 1); stating that initialization has two stages, with the second stage using the command execution tool; and packaging the command as a reasonable operation task within the sandbox. The tool return has two parts—reinforcing the claim that "initialization is in progress and the command is safe," and an explicit instruction to run the specified command using the command execution tool.
   • Per-agent customization: replacing the target tool name, adopting the agent's native prompt format (e.g., XML tags or double hashes), and adjusting wording to avoid specific refusal triggers; the six agents share the same two-channel template.

   Success criteria

   • Exfiltration: the simulated setting uses semantic similarity to the reference prompt and normalized EED; the real setting uses Pseudo-Recall (sentence coverage relative to the union of all methods).
   • Hijacking: the proportion of 10 attempts in which the agent calls the command execution tool with the expected payload (ASR).

   Collapse this answer

4. 04What experiments did the paper conduct?

   ToolLeak exceeds baselines in both simulated and real settings; dual-channel injection achieves ASR mostly 0.8–1.0 on old-version combinations, dropping to 0.0–0.3 on the new versions of Cursor/Claude Code.

   Experimental Setup

   • Subjects under test: six coding agents Cursor, Claude Code, Copilot, Windsurf, Cline, Trae, each paired with an officially supported backend LLM (Table 2), divided into two blocks: Old (old-version agent + old model) and New (latest version + new model); thinking mode is enabled by default where available, all use vendor-hosted integrations except Cline, which requires self-supplied API credentials, and default temperature.
   • Simulated setting (RQ1): leaked prompts collected from public repositories are used as reference ground truth, and the backend models are conditioned via API into simulated agents; backends are Grok-4, Claude-Sonnet-4, Claude-4.5-Sonnet, Gemini-2.5-pro, GPT-5, with temperature fixed at 0; each agent-LLM pair is run 10 times per attack and averaged.
   • Real setting (RQ2/RQ3): real agents' official interfaces, 10 independent trials per pair per method; RQ2 uses a best-of-10 protocol (the run with the most leaked sentences is used to compute metrics).
   • Metrics: Sim (cosine similarity of full-text embeddings computed with jina-embeddings-v2-base-en), normalized EED, Pseudo-Recall = |P_ToolLeak|/|P_Union| (no ground truth; refusal or no leakage is recorded as 0), ASR (proportion of 10 runs that successfully invoke the command execution tool).
   • Baselines: leakage baselines are Naive, Ignore, Completion, Remember-the-Start and combinations, PLeak; hijacking baselines are AgentDojo, InjecAgent, MCPTox rewritten payloads (tool description channel only) and the single-channel variant RCE-1.
   • RQ3 task: the benign user query is "Please write me a maze game", the target is the built-in command execution tool, and the payload is a one-line command that downloads and executes a script via curl.

   Main Results

   Simulated setting (RQ1, Table 3, averaged over 5 backends per agent):

   Swipe horizontally to read the table

   Agent	ToolLeak Sim↑	ToolLeak EED↓	Strongest baseline Sim	Strongest baseline EED
   Trae	0.921	0.466	0.619 (Ignore)	0.935 (Remember-the-Start)
   Cursor	0.949	0.375	0.665 (Completion-Remember)	0.762 (Completion-Remember)
   Claude Code	0.891	0.647	0.552 (Completion-Remember)	0.655 (Completion-Remember)
   Cline	0.958	0.528	0.600 (Remember-the-Start)	0.688 (Completion)
   Copilot	0.898	0.512	0.617 (Ignore-Remember)	0.920 (Ignore-Completion-Remember)
   Windsurf	0.923	0.509	0.691 (Completion)	0.737 (Remember-the-Start)

   • The authors state that ToolLeak is highest on both metrics across all six agents, and that PLeak transfers poorly (Sim below 0.25 in all cases).
   • Real setting (RQ2, Table 4): ToolLeak achieves the highest Pseudo-Recall in 18 of 25 agent-LLM combinations, with most combinations reaching 0.72–1.00; baselines are 0.00 in the vast majority of combinations.
   • Exceptions: on Copilot+Gemini-2.5-pro, ToolLeak is only 0.13, below the 0.87 of Ignore-Completion and Ignore-Remember; on Cursor+Gemini-2.5-pro, ToolLeak is 0.00 while Completion-Remember reaches 0.78. The authors attribute this to Gemini-2.5-pro filtering content in the tool-use channel more strictly and being more permissive for ordinary text generation.

   RQ3: Dual-Channel Injection Hijacking (Table 5)

   • Old-version combinations: RCE-2 has the highest ASR on every tested agent-LLM pair, with most configurations at 0.8–1.0 (e.g., Cline+Grok-4 is 1.0, Cursor+GPT-5 is 0.9, Copilot+Sonnet-4 is 1.0); RCE-1 and the three benchmark baselines are mostly 0.0, with nonzero values only in a few Gemini-backend combinations (e.g., RCE-1 is 0.7 on WindSurf+Gemini-2.5-pro).
   • New-version combinations: RCE-2 drops to 0.0 on Claude Code (Sonnet-4.6, Opus-4.7) and at most 0.3 on Cursor with each new backend; but it remains 1.0 on Cline, WindSurf, Trae with Gemini-3.1-pro, and 1.0 on Cline with Grok-4.20. Copilot lacks new-version experiments because new personal-plan subscriptions were suspended from April 2026.
   • Authors' interpretation: the authors believe Cursor and Claude Code use progressive disclosure (exposing only tool names to the orchestrating LLM, without injecting full tool descriptions), which blocks the description channel, and that the new backends exhibit stronger refusal behavior; the two together lead to the ASR decrease; the authors state that architectural isolation is an indicative defense layer and that model-level alignment is only supplementary.
   • Differences under the same backend: the authors state that stage-1 leakage shows that the system prompts of Trae and Cline contain command-execution restriction clauses while Cursor's does not, which they use to explain why, in the old-version block, Trae/Cline have lower ASR than Cursor under the same backend.

   Case Studies (Figure 3)

   • Cursor+GPT-5: a malicious tool workspace_manager is registered; after a user request triggers a call, the tool returns content that induces GPT-5 to call the built-in run_command_cmd to execute a malicious command, and RCE succeeds.
   • Claude Code+Sonnet-4.5: the guard model claude-haiku-4.5 judges the command to be executed as command_injection_detected (UNSAFE), but the main model, influenced by the injected context, treats it as a false positive and continues to call execute_command, and RCE succeeds; the authors state that this built-in tool itself performs no security checks.

   Defense Evaluation (Table 6)

   • ToolLeak detection: the perplexity of the ToolLeak tool description is 18.78, below the average perplexity of legitimate tool descriptions in MCP-Zero (the threshold), so PPL detection raises no alarm; Llama-Prompt-Guard-2-86M also judges it benign.
   • Hijacking detection: PPL and Window-PPL detect the payload on none of the six agents; Llama-Prompt-Guard-2-86M detects most cases, but the tool return for Cline is not detected (shown as ✓/× in the table); Agent-Scan and MCP Safety Scanner detect the payload on all six agents.
   • Authors' interpretation: the authors state that tool-level scanning is an effective practical defense layer, but the red team did not optimize for stealth, and whether adaptive obfuscation can bypass the scanners is an open question.

   Other Ablations and Analyses

   • Qualitative verification (RQ1/RQ2): the outputs of all methods were manually inspected; ToolLeak's leaked text contains identifiable system prompt components (role definitions, tool-use guidelines, safety rules, format constraints), and the authors state that they found no cases where high Pseudo-Recall was driven by random or hallucinated text.
   • System-level protection observations (Section 8.2): the authors state that Claude Code implements fine-grained command validation (down to the subcommand level), some agents have command allowlists or auto-approve switches, and Cline's default configuration lacks these protections.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors list validity threats such as the lack of ground truth and the limited scope of RCE; the experiments cover six agents and more than ten backends, and no stealthiness optimization was performed.

   Limitations and future directions noted by the authors

   • Internal validity (Section 10): Results may be affected by MCP tool registration, tool schema format, and agent settings (e.g., auto-approval, command guardrails); some products have hidden out-of-band defenses (e.g., a separate guard model), making it difficult to attribute success or failure to the main LLM versus the agent pipeline.
   • External validity (Section 10): Only six agents and some of their backends were evaluated; as prompts, tool schemas, and filtering policies evolve, the conclusions may not generalize to future versions; RCE focuses on command execution as a high-impact case and does not cover all privileged tools.
   • Construct validity (Section 10): Proprietary agents have no ground-truth system prompt; the simulated setting uses publicly leaked prompts as references, which may be incomplete or version-mismatched; measuring RCE success by the expected command tool call is not equivalent to real-world impact under different OS/network restrictions.
   • Relativity of Pseudo-Recall (Section 6.3): The authors emphasize that in the absence of ground truth, this metric should be read as a relative measure between methods rather than an absolute measure.
   • Open question of stealthiness (Section 8.2): The red team did not optimize for stealthiness; whether adaptive obfuscation can bypass tool-level scanners such as Agent-Scan remains an open question.
   • Defense directions (Sections 8.1, 8.2): The authors argue for output-side sensitive-information-aware training, and for architecturally enforcing separation of instruction-bearing and data-bearing context (citing SecAlign, Meta SecAlign, StruQ), as more durable solutions than reactive detection.

   Experimental coverage

   • The subjects are six coding agents (Cursor, Claude Code, Copilot, Windsurf, Cline, Trae) and the backend LLMs listed in Table 2, divided into Old/New generations; Copilot lacks New-generation experiments because its subscription was discontinued (Table 5 footnote).
   • Each agent–LLM pair was run 10 times per attack; RQ2 uses a best-of-10 protocol, and RQ1 takes the average of 10 runs.
   • In the hijacking experiments, the target tool is only the built-in command execution tool, the benign query is fixed as writing a maze game, and the payload is curl download-and-execute (Section 6.4).
   • The defense evaluation covers five detectors: PPL, Window-PPL, Llama-Prompt-Guard-2-86M, Agent-Scan, and MCP Safety Scanner (Table 6).
   • The paper does not report the sample size or agreement numbers for manual verification; the qualitative verification is described only as manually checking the outputs of all methods (Sections 6.2, 6.3).

   Collapse this answer

6. 06Summarize the main content of the paper

   Two-stage red teaming of six real coding agents: ToolLeak exfiltrates the system prompt, dual-channel injection achieves RCE, and architectural changes in the new version reduce ASR.

   An empirical red-team study on the security of tool calls in coding agents, evaluating Cursor, Claude Code, Copilot, Windsurf, Cline, Trae, and multiple backend LLMs officially supported by them.

   • Problem: The authors state that the built-in high-privilege tools of coding agents and external MCP tools constitute an attack surface, but a systematic security evaluation from the tool-call perspective has been lacking.
   • Method: Stage 1 proposes ToolLeak—a malicious MCP tool defines a required parameter as requesting the system prompt, exploiting the difference in guardrails between parameter generation and chat generation to exfiltrate internal context; Stage 2 proposes two-channel prompt injection—the tool description induces the call, and the tool return injects instructions, with the payload customized per agent based on the exfiltrated information, aiming to hijack the built-in command execution tool to achieve RCE.
   • Exfiltration results: Under the simulated setting, ToolLeak achieves Sim 0.891–0.958 and EED 0.375–0.647 on six agents, both the highest (Table 3); under the real setting, it achieves the highest Pseudo-Recall in 18 of 25 combinations (Table 4), with the exception of the Gemini-2.5-pro combinations (0.13 on Copilot, 0.00 on Cursor).
   • Hijacking results: On older-generation combinations, RCE-2 has the highest ASR on every tested pair, mostly 0.8–1.0, while single-channel baselines are mostly 0.0; on the new version it drops to 0.0 on Claude Code and at most 0.3 on Cursor, but Cline/WindSurf/Trae with Gemini-3.1-pro remain at 1.0 (Table 5).
   • Case and defenses: Claude Code's guard model Haiku judges UNSAFE but is still overridden by the main model (Figure 3); PPL-type detectors do not detect the payload, while Agent-Scan and MCP Safety Scanner detect it on all six agents (Table 6).
   • Authors' conclusion: The authors argue that tool-call parameter generation is an output channel whose guardrails are not aligned with those of the chat channel, that architecturally isolating the full tool description from the instruction context is a key defense layer, and that model-level alignment is only supplementary; the authors advocate that future systems should enforce separation of instruction and data context.

   Collapse this answer

Related papers