DecodingTrust-Agent Platform (DTap): A Controllable Interactive Red Teaming Platform for AI Agents

Core question: how to evaluate the safety risks of AI agents under adversarial conditions at scale in realistic, controllable, reproducible environments.

• Problem setting: Agents call external tools, process heterogeneous data, and execute long-horizon high-risk operations in dynamic, untrusted environments; the authors state that existing real-world incidents show attackers can manipulate agents into leaking API keys, deleting user data, or initiating unauthorized transactions.
• Existing gaps: The authors state that existing agent red-teaming work has limited attack surfaces (e.g., only direct prompt injection) or relies on simplified static environments, and cannot cover the full spectrum of real risks; the bottleneck is the lack of a fully controllable yet realistic evaluation environment.
• Core idea: Replicate the official MCP and GUI of real systems (e.g., Google Workspace, PayPal, Slack) 1:1, and support dynamic state, deterministic transitions, and arbitrary state reset in a sandbox, thereby safely injecting and evaluating adversarial scenarios.
• Paper outputs: Three parts—the simulation platform DTAP (14 domains, 50+ environments), the automated red-teaming agent DTAP-Red (200+ attack strategies), and the benchmark DTAP-Bench (6,682 tasks, 3,876 red-teaming tasks, each with a verifiable judge).
• Threat models:
  • Indirect threat model: A benign user issues a legitimate request, and a third-party attacker introduces hidden malicious goals through tool, skill, and environment injection, without using direct prompt injection.
  • Direct threat model: The user themselves acts as the attacker and directly issues malicious queries, possibly combined with indirect injection.
  • Attacker knowledge: The authors state they assume generic black-box access to the agent.
  • Victim systems: Tool-using agents that support MCP, such as agents built with frameworks including OpenAI Agents SDK, Claude Code, Google ADK, and OpenClaw.

Agent red-team environments and simulators

• AgentDojo [18], AgentHarm [6] — the authors say they lack dynamic environments, rely on static synthetic tools with injections hard-coded in outputs, and cannot capture the risks arising from real agent dynamics.
• SHADE-Arena [31], Petri [23] — they introduce more interactive environments, but the authors say they remain highly simplified and cannot faithfully model the diversity of real adversaries.
• ST-4 [33], RedTeamCUA [35], SafeArena [60], ShieldAgent [12] — they evaluate agents in more realistic settings, but the authors say the number of environments and domains is small, and they often lack design properties needed for reproducible red teaming, such as flexible state reset and deterministic transitions.

Agent red-team benchmarks

• AgentDojo [18] — the authors say it focuses on unrealistic tool-output injection (appending malicious instructions directly after the output).
• AgentHarm [6], ST-Web [33], SafeArena [60] — they mainly study direct prompt injection, and the authors say they do not use advanced attack techniques.
• More agentic attack work [81, 14] — the authors say they often assume a specific domain or white-box attacker capability, limiting their applicability in realistic threat settings; they also note that these benchmarks generally lack a systematic risk taxonomy and rely on a small number of ad hoc malicious goals.

Sources of the attack algorithms used in this paper

• DTAP-Red's attack algorithm library includes existing jailbreak and injection methods; the paper names GCG [87], ArtPrompt [30], flip-attack, base64 encoding, etc.

Baselines and comparison: this paper does not compare attack performance with other methods; instead, Table 2 compares platform features (number of environments, injection types, threat models, verifiable judge, etc.), and DTAP is checked on all listed items; Figure 5 gives a t-SNE comparison of Claude Code agent trajectories on DTAP and AgentDojo.

Authors' positioning: the authors say DTAP is the first controllable interactive agent red-team platform, and DTAP-Bench is a policy-driven benchmark covering 4K malicious goals extracted from 60+ real security policies, considering previously unexplored injection vectors such as environment, skill, tool and prompt, and assuming generic black-box access to the agent.

Overall approach: first simulate real systems' agent interfaces 1:1 in the DTAP platform, then have the automated red-teaming agent DTAP-Red iteratively optimize attacks against malicious goals extracted from policies, and solidify successful attack sequences into DTAP-Bench, using deterministic rules to determine whether an attack succeeds.

DTAP platform design

• Environment replication: 50+ environments replicate the official GUI and MCP tool interfaces of their real counterparts (tool names, descriptions, input schemas, and return formats correspond one-to-one); where official ones are missing, widely used community implementations are used; all actions are controllable and reversible in the sandbox.
• Four design properties: dynamic stateful interaction (retaining intermediate state for verification), deterministic environment transitions (replayable), flexible reset by design (instantiable to any state snapshot), and containerized multi-tenant isolation for parallel evaluation.
• Agent integration: plug-and-play integration of any MCP-supporting agent through a unified MCP interface; built-in support for OpenAI Agents SDK, Google ADK, LangChain Agents, Claude Code, OpenClaw, etc.

DTAP-Red red-teaming agent

• Closed-loop three stages: attack optimization and generation → executing the attack against the victim agent → verifiable judge evaluation and iterative improvement; the only inputs are the malicious goal and the corresponding verifiable judge.
• Attack skill library: two categories—attack algorithms (jailbreak/injection methods such as GCG and ArtPrompt, which transform malicious instructions into more covert forms and can optimize payloads on the fly in a controlled environment) and injection strategies (knowledge of how to construct injection content and choose injection points).
• Injection action space: covers user prompt, tool descriptions, skill metadata and body, and environment data sources; supports modes such as prefix, suffix, and overwrite, and can be combined across multi-step temporal sequences; each environment is equipped with an injection MCP server, and each tool corresponds to a preconfigured injection entry point.
• Multi-layer memory module: organizes successful and failed experience in three layers by risk category, malicious goal, and threat model; uses an ϵ-greedy strategy to balance exploration and exploitation.
• Iterative optimization: after each attack round, the verifiable judge directly checks the environment state to determine success or failure; on failure, the attack refinement judge analyzes the injection sequence and victim trajectory and gives feedback (e.g., the injection was not observed, or was explicitly refused), iterating until success or budget exhaustion.

Malicious goals and judging

• Policy-driven: risk categories are extracted from domain policies (e.g., Salesforce AUP), regulations (e.g., EU AI Act, GDPR, FINRA), and use-case-driven risks, and malicious goals are then manually designed.
• verifiable judge: each task is equipped with judge.py, which uses a rule checker to directly check the final environment state (e.g., whether the recipient, amount, and status in PayPal are all as expected), distinguishing full success/partial success/failure; the authors state that no LLM judge is used, to avoid instability and reward speculation.

DTAP-Bench construction

• Three subsets: benign task set (utility evaluation), indirect red-teaming task set, and direct red-teaming task set; 6,682 tasks in total, of which 3,876 are red-teaming tasks, each corresponding to a unique malicious goal (Table 6).
• Attack solidification: the attack sequences for red-teaming tasks are obtained by DTAP-Red optimizing against a surrogate agent (e.g., OpenAI Agents SDK + GPT-5.1) within a budget of at most 10 iterations, and then manually filtered to ensure a balanced distribution of injection types; all agents are evaluated on the same set of zero-shot transferred attacks.
• Seed data: a persona-based synthesis pipeline samples user attributes and generates benign data consistent with the profiles (e.g., personalized email threads) as the initial environment state.
• Task structure: each task contains config.yaml (task instruction, agent configuration, injection sequence), setup.sh (reset and seed the environment), and judge.py (full examples are given in Appendix A.2 and A.3).

Experimental setup

• Agents tested: 8 configurations across 4 frameworks — OpenAI Agents SDK (GPT-5.4, GPT-5.2, GPT-OSS-120B), Claude Code (Sonnet-4.5), Google ADK (Gemini-3-Pro), OpenClaw (GPT-5.5, GPT-5.2, DeepSeek-V4-Pro).
• Benchmark: DTAP-Bench, containing 1,998 indirect red-team tasks, 1,878 direct red-team tasks, 2,806 benign tasks, covering 14 domains (Table 6).
• Attack source: attacks are optimized by DTAP-Red against the proxy agent (OpenAI Agents SDK + GPT-5.1) within at most 10 iterations, and transferred zero-shot to the tested agents; the paper does not report DTAP-Red's own backbone model.
• Metrics: ASR (proportion of malicious goals fully achieved, reported separately by threat model) and BSR (proportion of benign tasks fully completed).
• Judging method: each task has a rule-based verifiable judge that directly checks the final environment state; no LLM judge is used.
• Number of repetitions: not reported in the paper.

Main results

According to Table 3 (ASR%) and Table 4 (BSR%), Overall column:

Swipe horizontally to read the table

Agent (framework + model)	Indirect ASR	Direct ASR	BSR
OpenAI Agents GPT-5.4	40.0	51.0	85.3
OpenAI Agents GPT-5.2	46.7	58.8	80.5
OpenAI Agents GPT-OSS-120B	28.5	46.5	36.2
Claude Code Sonnet-4.5	25.2	26.9	80.8
Google ADK Gemini-3-Pro	55.7	47.9	87.0
OpenClaw GPT-5.5	17.7	28.9	86.3
OpenClaw GPT-5.2	35.6	38.6	78.5
OpenClaw DeepSeek-V4-Pro	41.7	59.6	83.3

• The authors say Google ADK (Gemini-3-Pro) is the most vulnerable under the indirect threat model, and OpenClaw (DeepSeek-V4-Pro) has the highest ASR under the direct threat model; even the most robust, Claude Code (Sonnet-4.5), still has ASR above 25% under both threat models.
• The authors say GPT-OSS-120B is both unsafe and insufficiently capable: BSR is only 36.2%, yet direct ASR is still 46.5%.
• The authors report that when malicious goals are input directly without optimized attacks, almost all agents refuse, with ASR near zero (e.g., Table 10, Table 15), which they take to indicate that the malicious goals themselves are severe and that attack effectiveness comes from DTAP-Red's optimization.

Injection vector analysis

According to Table 5 (ASR% for each injection type and combination, across all red-team tasks):

• Single-facet comparison: ASR for skill and tool injection is generally higher than for env injection, e.g., on GPT-5.4, skill 83.3% vs env 22.2%; the authors believe environment inputs are treated as more external, while internalized channels are more easily exploited.
• Framework differences: OpenClaw's tool injection ASR (21.1% for GPT-5.5) is lower than Google ADK's (63.4%); the authors say OpenClaw's trust calibration toward external plugins is stronger.
• Combination amplification: combined injection often exceeds a single channel, e.g., on GPT-5.2, tool 50.1% → env+tool 65.6%; the authors say multi-facet attacks more easily manipulate agents.

Failure modes and harness analysis

• execute-then-refuse: the authors say they frequently observe OpenAI Agents SDK and Google ADK first executing harmful tool calls and then refusing (Appendix B, C, L; figure in Appendix C.6.2 Figure 28), and speculate this is related to batched tool calls hindering per-tool consequence reasoning; Claude Code and OpenClaw make sequential calls.
• Role of the harness: the authors report that with the same GPT-5.2 backbone, switching from OpenAI Agents SDK to OpenClaw lowers indirect ASR from 46.7% to 35.6% and direct from 58.8% to 38.6%, at a BSR cost of 2%; the authors believe harness design plays a dominant role in robustness.
• Risk category differences: the authors say context-dependent risks (Data Exfiltration, Dangerous Actions, etc.) have higher ASR, while content-level risks (General AI Restrictions, Fraud/Impersonation) have lower ASR, possibly due to model-level alignment.

Other ablations and analyses

• DTAP-Red efficiency (Figure 8, CRM domain): for most risk categories, attack success is reached in an average of 2–5 iterations, with Fraud & Impersonation taking at most 6.3 (indirect); ASR rises from about 37% at iteration 1 to over 85% within 3 iterations, and approaches 100% after 10.
• Attack skill effects (Figure 9, CRM domain): multi-turn fake conversation and general-misinformation have the highest ASR, above 50%; skills are complementary across risk categories.
• Task complexity (Figure 7): in most domains the minimum number of tool calls per task exceeds 10, generally highest for indirect red-team tasks.
• Environment differences (Figure 13, Workflow domain): under the direct threat model, WhatsApp 80.00 and Google Docs 77.78 are highest, while PayPal is only 11.11; the authors say communication-type environments are more easily exploited and financial ones less so.
• Exception: GPT-OSS-120B has low indirect ASR (6.1 in the Workflow domain) but high direct ASR (62.1); the authors report that its benign task completion rate in indirect tasks is only 40.2% (43/107), and believe the low indirect ASR mainly reflects weak execution capability rather than greater safety.

Limitations and future directions noted by the authors

• The paper does not specifically discuss limitations (there is no Limitations section, and the Conclusion and the provided portion of the appendix list no limitations or future work statements).

Scope of experimental coverage

• Agents tested: 8 configurations across 4 frameworks — OpenAI Agents SDK (GPT-5.4, GPT-5.2, GPT-OSS-120B), Claude Code (Sonnet-4.5), Google ADK (Gemini-3-Pro), OpenClaw (GPT-5.5, GPT-5.2, DeepSeek-V4-Pro) (Sec. 6.1).
• Task scale: 1,998 indirect red-team tasks, 1,878 direct red-team tasks, 2,806 benign tasks, covering 14 domains (Table 6).
• Attack transfer setting: attacks are optimized by DTAP-Red against the proxy agent OpenAI Agents SDK + GPT-5.1 within at most 10 iterations, and the tested agents are evaluated by zero-shot transfer; Appendix B.4 states that the 100.0% ASR in the GPT-5.1 row is a matched attack-generation upper bound and is not a comparable robustness result.
• Judging method: all are rule-based verifiable judge checks of environment state, with no LLM judge; the paper does not report agreement between judges and humans.
• Unreported information: DTAP-Red's own backbone model, the number of repeated runs per task, and confidence intervals or statistical tests for ASR are all unreported; DTAP-Red's efficiency and skill ablations are given only in the CRM domain (Figure 8, Figure 9).

This is an agent safety evaluation paper: the authors build a red-teaming platform DTAP, an automated red-teaming agent DTAP-Red, and a benchmark DTAP-Bench, to evaluate the safety risks of AI agents at scale in realistic simulated environments.

• Problem to solve: The authors state that existing agent red-teaming environments are static with a single attack surface, and lack the capability for realistic, controllable, reproducible large-scale risk evaluation.
• Method highlights: DTAP replicates the GUI and MCP interfaces of 50+ real systems (Gmail, PayPal, Slack, etc.) across 14 domains 1:1, supporting dynamic state, deterministic replay, and arbitrary reset; DTAP-Red iteratively optimizes attacks on the prompt, tool, skill, environment, and combined injection surfaces, and uses a rule-based verifiable judge that checks environment state to determine success or failure; DTAP-Bench contains 6,682 tasks, with malicious goals drawn from 300+ risk categories across 60+ safety policies.
• Main results: Among 8 agent configurations, the highest indirect ASR is Google ADK (Gemini-3-Pro) at 55.7%, and the highest direct ASR is OpenClaw (DeepSeek-V4-Pro) at 59.6%; the lowest, Claude Code (Sonnet-4.5), still reaches 25.2%/26.9% (Table 3).
• Injection surface findings: skill and tool injection ASR is higher than env injection (e.g., 83.3% vs 22.2% on GPT-5.4), and combined injection further amplifies it (Table 5).
• Failure modes: The authors state that OpenAI Agents SDK and Google ADK frequently execute the harmful call first and then refuse; switching skeletons with the same backbone can reduce ASR by about 20 percentage points at a BSR cost of 2%.
• Authors' conclusion: The authors believe existing defenses are insufficient to support real deployment and that holistic, environment-aware safety design is needed; the attacks are zero-shot transfers after optimization against a GPT-5.1 agent, and the paper does not specifically discuss limitations.