SkillTrojan: A Backdoor Attack Against Skill-Based Agent Systems

1. 01What problem does this paper attempt to solve?

   The authors propose SkillTrojan: a backdoor attack targeting the skill abstraction layer, hiding malicious logic in reusable skill implementations rather than in model parameters or prompts.

   Core problem: In skill-based agent systems, reusable skill implementations are an unchecked backdoor attack surface.

   • Setting and importance: The authors state that the skill abstraction has become a common design pattern in agent systems; skills encapsulate process logic, tool calls, and execution workflows, and persist across tasks, users, and deployments; skills are typically treated as trusted modules, and their internal behavior is rarely audited the way model outputs are.
   • Shortcomings of existing methods: The authors state that prior backdoor research on LLM agents mainly manipulates control channels (models, prompts and planning context, tool/memory interfaces), and that evaluations and defenses focus on model-observable behavior or isolated interaction surfaces, leaving the skill implementation control point uncovered.
   • Core observation: Compromising a widely adopted skill can affect many downstream runs without changing the underlying model, prompts, or toolset; malicious logic can be hidden in a seemingly reasonable skill and activated through standard skill composition.
   • Proposed in the paper: The SkillTrojan backdoor attack framework (encrypted payload sharding, trigger activation, automatic synthesis of backdoored skills from arbitrary templates), and the SkillTrojan dataset of 3,000+ backdoored skills.
   • Threat model:
     • Goal: Maintain normal task behavior on clean inputs (stealth), and on trigger inputs cause normal skill execution to lead to payload P running (reliable activation).
     • Knowledge: The paper does not state whether the attacker knows the internals of the victim model; it assumes the underlying language model and its training data are trusted and unmodified.
     • Capability: Publish and distribute skill packages that appear harmless and improve task performance, inducing developers to adopt them; after installation, no access to the host environment, OS permissions, or model internals, interacting only through user queries.
     • Victim system: Skill-based agents that can dynamically load third-party skills from public repositories or marketplaces; excludes cross-run state and direct compromise of the base model/remote services/operating system.

   Collapse this answer

2. 02What related research exists?

   Related work falls into two categories: coding agents and executable skill abstractions, and backdoor attacks against models and agents; this paper is positioned at the execution layer.

   Coding agents and executable skill abstractions

   • Deng et al. (2025), Liu et al. (2025), et al.—coding and tool-execution agents solve tasks by composing executable tools, scripts, and workflows, using modular abstractions such as skills/tools/actions to encapsulate reusable code and API calls.
   • The authors point out that this line of work mainly focuses on improving agent capability, planning efficiency, and compositional generalization; once a skill is installed it is assumed to be benign, and security analysis focuses on model outputs, prompts, or high-level planning behavior.

   Backdoor attacks against models and agent systems

   • Li et al. (2025a), Wu et al. (2025), Yu et al. (2025)—traditional backdoor attacks directly target models: training data poisoning, parameter manipulation, trigger-based input distributions, with the model as the primary control surface.
   • Xu et al. (2024), Zhu et al. (2025), Chen et al. (2024), Feng et al. (2026)—extend backdoors and adversarial attacks to agent systems, including prompt injection, malicious tool descriptions, memory poisoning, and manipulation of agent control logic; the authors point out that these methods act on transient interaction surfaces such as prompts, tool calls, and memory entries, assuming malicious behavior is triggered within a single agent episode.
   • Zhang et al. (2024), Cheng et al. (2025; 2024), Zheng et al. (2025a)—the authors state that existing evaluations and defenses mainly monitor model inputs, outputs, or isolated interaction channels.

   Baselines and benchmarks

   • The comparison baselines are GCG, AutoDAN, CPA, BadChain, and AgentPoison, adapted to the same skill-based agent setting; the evaluation benchmarks are EHR SQL (main benchmark) and SWE-Bench Verified (additional benchmark, results in Appendix B).

   Positioning of this paper: The authors state that SkillTrojan targets the execution layer of coding agents, embedding backdoors directly into reusable executable skills that persist across tasks and deployments and are activated through normal skill composition, without modifying the underlying model, and therefore falls outside the scope of existing backdoor defenses.

   Collapse this answer

3. 03How does the paper address this problem?

   SkillTrojan splits the encrypted payload into N fragments hidden across multiple skill calls; upon triggering, a verification action checks them as a set, reconstructs the payload, and executes it.

   Overall approach: Given a benign skill template s = (m, A), an attacker-specified payload program P, and a trigger predicate Trigτ, SkillTrojan generates a backdoored skill s̃ = (m̃, Ã); non-trigger queries retain the original functionality, while under trigger queries normal skill execution reconstructs P and executes it as a side effect.

   Payload encoding and skill instrumentation

   • Encryption and fragmentation: The payload is encrypted with symmetric encryption C = Enc(P; k), then encoded and split into N fragments; each fragment c_j = ⟨ j, seg_j(C) ⟩ carries an index identifier, supporting order-agnostic reconstruction.
   • Role of fragment count N: N is a tunable parameter controlling the trade-off between stealth and activation robustness; a larger N means less information per fragment and lower risk of accidental activation under partial execution, but requires a more complete execution trace for reconstruction.
   • Specification modification: m is changed to m̃, injecting conditional control logic parameterized by Trigτ(q); when not triggered, m̃ is semantically equivalent to m; when triggered, it biases the agent toward an execution trace covering the specified action set, but does not prescribe a fixed execution order or explicit malicious instructions.
   • Artifact instrumentation: Modified actions retain their original functional behavior and output; when triggered, fragments are written to a local intermediate channel in the skill execution context (instantiated as seemingly normal fields in tool output, which are recorded in the tool-call trace); when not triggered, no fragments are emitted; fragments are limited to a single execution and do not persist across queries.

   Triggered reconstruction and execution

   • Set verification: The specified verification action checks whether all fragments have been collected, with the criterion being set inclusion {c₁, …, c_N} ⊆ F(q), independent of execution order, timing, or control-flow structure.
   • Reconstruction and execution: After verification passes, the ciphertext is concatenated by index and decrypted P = Dec(c₁ | ⋯ | c_N; k), and executed in the skill execution environment as a side effect of normal task completion; intermediate artifacts are deleted after execution.
   • Decoupling from the response: Payload execution is independent of the agent's response generation; the agent gives correct and benign output to the user, and malicious behavior occurs only through execution-side side effects.

   Dataset construction (SkillTrojanX)

   • Template sources: The top 1,200 skills are crawled from public skill repositories/marketplaces by popularity, retaining packages that contain a parseable specification file and executable artifacts, and normalized into a unified representation.
   • Variant generation: Given an attacker configuration (P, τ, N), trigger strings/patterns, payload families, encryption/encoding schemes, and fragment count N are varied, generating multiple variants per template.
   • Metadata: Records trigger identifiers, payload families, encryption/encoding variants, fragment count N, and the action sets designated for fragment emission and verification.

   Success criteria and default parameters

   • Metrics: Task accuracy ACC on clean queries and attack success rate ASR on trigger queries; ASR is determined by the execution marker returned by the verification tool and corroborated by deterministic side effects.
   • Default settings: Unless otherwise stated, poisoning ratio ρ = 0.2, fragment count N = 3, XOR+Base64 for encryption/encoding; poisoned queries are constructed by inserting a trigger phrase into the original natural language query.

   Collapse this answer

4. 04What experiments did the paper conduct?

   Evaluating 9 models and 5 baselines on EHR SQL, SkillTrojan achieves 97.2% ASR and 89.3% ACC on GPT-5.2-1211-Global.

   Experimental setup

   • Models tested: Open-Weight: GLM-4.7, Qwen3-Coder, GLM-4.6, Qwen3-VL-235B-A22B-Instruct; Closed-Weight: GPT-4o-Mini-0718-Global, Claude-Haiku-4.5, Claude-Sonnet4.5, Qwen3-Max, GPT-5.2-1211-Global; models serve as the agent's policy model, with skill implementations and evaluation protocols kept consistent across models.
   • Benchmarks: The main benchmark is EHR SQL (natural language clinical queries composed into SQL and executed against an EHR database); the additional benchmark is SWE-Bench Verified (results in Appendix B); cross-framework evaluation uses a separate industry-oriented coding benchmark (Appendix G).
   • Baselines: GCG, AutoDAN, CPA, BadChain, and AgentPoison adapted to the same skill-based agent setting, using the same trigger insertion process, poisoning ratio, and tool environment; two additional controls are Non-attack (no skill stack) and Skills (Native) (a backdoor-free version of the same skills).
   • Metrics and criteria: ACC is the proportion of clean queries judged correct by LLM-as-a-judge (judge prompt in Appendix D; the paper does not state which model the judge uses); ASR is the proportion of poisoned queries in which the payload is successfully reconstructed and executed, determined by the verification tool's execution marker and deterministic side effects.
   • Default parameters: ρ = 0.2, N = 3, XOR+Base64; the paper does not report the number of query samples or repetitions per condition.

   Main results

   SkillTrojan versus each method on EHR SQL (from Table 1 and Table 2, ACC/ASR in %):

   Swipe horizontally to read the table

   Model	Non-attack ACC	SkillTrojan ACC	SkillTrojan ASR	Strongest baseline ASR (method)
   GLM-4.7	84.1	85.2	62.1	57.2 (AgentPoison)
   Qwen3-Coder	71.5	76.3	64.7	62.5 (AgentPoison)
   GLM-4.6	76.0	81.3	72.0	60.8 (AgentPoison)
   Qwen3-VL-235B-A22B-Instruct	53.2	48.4	26.7	37.6 (AgentPoison)
   GPT-4o-mini-0718-Global	71.6	68.5	54.3	53.7 (AgentPoison)
   Claude-Haiku-4.5	73.1	82.7	57.3	54.8 (AgentPoison)
   Claude-Sonnet4.5	86.5	90.7	64.2	56.2 (AgentPoison)
   Qwen3-Max	82.7	86.6	74.7	57.6 (AgentPoison)
   GPT-5.2-1211-Global	73.0	89.3	97.2	58.3 (AgentPoison)

   • Authors' interpretation: The authors state that SkillTrojan achieves high ASR in both regimes while largely maintaining clean ACC, with ACC higher than Non-attack in some settings; the authors explain this as installing (benign) skills itself improving task completion, rather than the backdoor logic.
   • Exception: On Qwen3-VL-235B-A22B-Instruct, SkillTrojan's ASR (26.7) is lower than AgentPoison's (37.6), and ACC (48.4) is also lower than Non-attack (53.2); the authors give no specific explanation for this model.
   • Authors' interpretation of baselines: The authors state that prompt/conversation-level baselines are unstable in tool-execution settings, with their success depending on the model choosing to follow the injected instruction pattern, while stronger models are better at refusing or are more robust to shallow prompt manipulation; BadChain's ASR drops to 8.3–14.2 on some Closed-Weight models (Table 2).

   Scanner defense analysis

   • What was tested: About 500 skills were randomly sampled, and Qwen3Guard-Gen-8B and Llama-Guard-3-8B were asked to classify each skill as safe or unsafe (Table 3).
   • Results: The proportion of SkillTrojan judged safe is 98.2% (Qwen3Guard) and 99.4% (Llama-Guard), versus 100.0% and 99.6% for Skills (Native); the unencrypted, unfragmented SkillTrojan w/o enc./seg. is judged safe only 3.4% and 2.8% of the time.
   • Authors' interpretation: The authors argue that simple LLM-based static scanning is insufficient to detect encrypted and fragmented skill-level backdoors, and that defenses need to reason about execution traces, provenance, permissions, and side effects.

   SWE-Bench Verified and cross-framework generalization

   • SWE-Bench Verified (Table 6): On Qwen3-Coder, SkillTrojan ACC 65.7, ASR 63.9 (Non-attack ACC 64.8); on GPT-5.2-1211-Global, ACC 70.4, ASR 92.8 (Non-attack ACC 70.6); on GLM-4.7, ACC 67.6, ASR 66.4 (Non-attack ACC 67.3); GCG's ASR is 10.3–26.1.
   • Cross-framework (Table 8, Appendix G): Across the four frameworks Claude-Code, IFlow, OpenHands, and OpenClaw and the five backbone models Claude Sonnet 4.5, GPT-5.1, Qwen3 Coder Plus, Qwen3.5 Plus, and GLM-5, ASR ranges from 63.53% to 100.00%; the authors note that these results come from a separate industry-oriented coding benchmark and should not be directly compared numerically with Tables 1–2.

   Other ablations and analyses

   • Fragment count N (Figure 5, GPT-4o-mini-0718-Global): The authors state that ASR shows a unimodal trend, highest at N=3, with ASR decreasing when N is too small or too large; ACC remains stable across N (only axes and legend are visible in the figure; the paper does not give specific values in the main text).
   • Encryption scheme (Table 5, GPT-4o-mini-0718-Global, N=3, ρ=0.2): XOR+Base64 and Hybrid have ACC of 68.5 and 68.4, and ASR of 54.3 and 54.6, respectively; the Heuristic-flag of a simple base64 detector drops from 78.0% to 21.0%.
   • Poisoning ratio ρ (Figure 6, Appendix A, GPT-4o-mini-0718-Global): ACC at ρ=0.5 remains close to the default 68.5%; ASR varies by no more than 3 percentage points within ρ ∈ {0.1, 0.2, 0.5}.
   • Clean-skill utility (Table 7, Appendix F): Skills (Native) has the highest clean ACC on the four open-source models (88.7/82.3/85.5/50.3), based on which the authors argue that the clean improvement comes from the skills themselves rather than the backdoor logic.
   • Tool-call distribution (Figure 3): The authors state that the number of tool calls in triggered runs rises slightly (corresponding to fragment emission and verification) but remains within the typical range for complex EHR queries; the figure shows two distributions, clean and triggered, along with axes, but specific values are not labeled.

   Collapse this answer

5. 05What are some points that can be further explored?

   The authors state that the future direction is defenses that jointly reason about skill composition, tool-call traces, and side effects; the experiments cover 9 main tested models and two benchmarks.

   Limitations and future directions noted by the authors

   • Defense directions (Conclusion): The authors state that an important direction for future work is developing targeted defense prompts, guardrail models, and runtime monitors that jointly reason about skill composition, tool-call traces, and side effects under real deployment constraints.
   • Motivation for defense mechanisms (Conclusion): The authors state that the results motivate defenses to explicitly model skill provenance and constrain execution-time side effects, including sandboxing, permissioned resource access, signed package provenance, static and dynamic code inspection, and execution trace auditing; these are directions proposed by the authors, and the paper does not implement or evaluate these defenses.
   • Conclusion on scanner insufficiency (Conclusion): The authors state that scanner analysis shows that simple lightweight LLM-based static screening is insufficient against encrypted and sharded skill-level backdoors.
   • Purpose of released artifacts (Impact Statement): The authors encourage future work to use the released artifacts for red-team evaluation, secure skill marketplace design, and defense research that inspects tool-call traces, side effects, and the provenance of executable skills.

   Scope of experiments

   • Models tested: 9 models in the main experiments (4 Open-Weight, 5 Closed-Weight); 3 models on SWE-Bench Verified; cross-framework evaluation of 4 frameworks × 5 backbone models (Table 8).
   • Benchmarks: EHR SQL (main), SWE-Bench Verified (Appendix B), and a separate industry-oriented coding benchmark (Appendix G, which the authors note is not directly comparable to the main results).
   • Defense evaluation: Only two scanners, Qwen3Guard-Gen-8B and Llama-Guard-3-8B, were evaluated, on a sample of approximately 500 randomly sampled skills (Table 3); the paper does not implement or evaluate defenses under adaptive attacks.
   • Ablation variables: Number of fragments N (Figure 5), encryption scheme (Table 5), poisoning ratio ρ (Figure 6), all on GPT-4o-mini-0718-Global.
   • Not reported in the paper: The number of query samples per condition, the number of experimental repetitions, the judge model used for LLM-as-a-judge, and the agreement between the judge and human judgments.

   Collapse this answer

6. 06Summarize the main content of the paper

   SkillTrojan hides encrypted payload fragments in reusable skills, reaches 97.2% ASR on GPT-5.2-1211-Global, and evades two LLM scanners.

   A paper on backdoor attacks against skill-based agent systems. The authors argue that reusable skill implementations are a blind spot in current agent security assumptions: skills persist across tasks and deployments, are treated as trusted components, and their internal behavior is rarely audited.

   • Method: SkillTrojan symmetrically encrypts an attacker-specified payload and splits it into N indexed fragments, instrumenting them into multiple seemingly harmless skill actions; fragments are emitted only when a trigger predicate is satisfied, and a verification action checks set inclusion to confirm all fragments are collected, then reconstructs, decrypts, and runs the payload as an execution-side side effect, while user-visible output remains correct and benign; it supports automatic synthesis of backdoored skills from arbitrary skill templates.
   • Main results: On EHR SQL, GPT-5.2-1211-Global reaches 97.2% ASR and 89.3% ACC (Non-attack ACC 73.0, Table 2); on the open-source side GLM-4.6 reaches 72.0% ASR and 81.3% ACC (Table 1); the exception is Qwen3-VL-235B-A22B-Instruct with 26.7% ASR, lower than AgentPoison's 37.6%.
   • Generalization: On SWE-Bench Verified, GPT-5.2-1211-Global reaches 92.8% ASR (Table 6); across four agent frameworks × five backbone models, ASR ranges from 63.53% to 100.00% (Table 8, separate benchmark).
   • Defense analysis: Qwen3Guard-Gen-8B and Llama-Guard-3-8B classify 98.2% and 99.4% of SkillTrojan samples as safe, respectively (about 500 sampled skills, Table 3); the authors argue that simple static scanning is insufficient against such backdoors.
   • Release: The SkillTrojanX dataset, containing 3,000+ backdoored skills generated from 1,200 templates, covering 6 template categories, 4 payload families, and 50 unique triggers (Table 4).
   • Authors' conclusion: As agent systems increasingly rely on reusable third-party skills, the security-critical surface expands from model behavior to the execution substrate in which skills reside; the authors state that this motivates defenses to explicitly model skill provenance, constrain execution-time side effects, and lists joint reasoning over skill composition, tool-call traces, and side effects as future directions for monitoring.

   Collapse this answer

Related papers