Supply Chain Poisoning Attacks on the LLM Coding Agent Skill Ecosystem

The core question is: can malicious logic in third-party skill documentation translate into actual system operations under model alignment and framework defenses?

• Use scenario: Skill packages provide tool invocation logic, context prompts, and configuration examples. The authors state that coding agents treat skill documentation as operational guidance, so risk can extend from text output to file, command, and network operations.
• Existing research boundary: The authors argue that ToolTweak, Skill-Inject, and PoisonedRAG mainly study tool selection or text poisoning and have not yet addressed the problem of skill documentation inducing code execution on the host system.
• Core observation: The authors argue that agents treat code examples in documentation as reference implementations, reusing and executing them in normal tasks; attacks therefore need not rely on separate explicit malicious instructions.
• Threat model: The scope is the post-loading stage, assuming the skill has been retrieved and loaded (§3).
  • Goals: O1 is system control and asset exfiltration; O2 is infrastructure poisoning in development, build, and deployment pipelines.
  • Knowledge: External attackers have no white-box access and cannot access the executor's system prompt.
  • Capabilities: Attackers construct and publish skills, controlling their metadata and execution content; they cannot tamper with user queries or bypass runtime isolation.
  • Victim system: Coding agents with file, shell, and network access capabilities, along with their execution environments.
• Proposed method: PoisonedSkills combines the DDIPE documentation embedding mechanism with an automated generation pipeline, separately counting malicious code generation and direct execution rather than treating both as host system compromise.

Related work focuses on how external content enters the model and which layer it affects once inside.

LLM supply chain security

• Training data and model backdoors: Carlini et al. (2024), Qu et al. (2025) discuss training data poisoning or code model backdoors; Gu et al. (2017)'s BadNets discusses backdoor risks in the model supply chain.
• Third-party applications and agent extensions: Greshake et al. (2023) study third-party applications manipulating LLMs through context; ToolTweak (2025) manipulates tool selection ordering, Skill-Inject (2026) studies skill file attacks. The authors describe the scope of the latter two as the tool selection or text generation layer.

Retrieval knowledge poisoning

• PoisonedRAG (2025): Injects adversarial text into knowledge bases to influence generated answers. The authors believe its threat model handles textual misinformation but does not model unauthorized operations that occur after skills connect to system tools.

Indirect prompt injection

• Liu et al. (2023) and Greshake et al. (2023): Study delivering instructions through external content, where attackers do not need direct control over user input.
• Markdown Image Injection (2024): Uses external content to induce data exfiltration. The authors summarize such payloads as imperative instructions and argue that DDIPE differs in that the payload can manifest as the code itself.

Experimental baselines and positioning of this paper

• Baseline methods: Use explicit natural language instruction injection, citing Liu et al. (2023) and InjecAgent (2024); the actual comparison uses this study's 1,070 skill samples, configured with Claude Code＋Claude Sonnet 4.6 (§5.1). No experiments on the InjecAgent dataset are reported.
• Author positioning: Focus on whether, after a poisoned skill has already entered the context, it can bypass alignment and framework defenses to induce action-space operations. The authors leave comparisons with more complex injection baselines to future work.

PoisonedSkills combines document embedding mechanisms with an automated generation pipeline: attack content enters the context as reference code or configuration, and the authors then check whether it is generated and executed.

Problem setup and success conditions

• Research boundary: Skills consist of metadata and execution content; metadata includes descriptions, code examples, and configuration templates. Retrieval and loading success is taken as a prerequisite and is not validated in the experiment (§3, Figure 2).
• Two conditions: The generation pipeline must produce skills containing disguised payloads; the agent must reproduce the payload and execute it through system interfaces (§4.1). Merely generating code and actual execution are counted separately in the evaluation.

Document-driven implicit payload execution

• DDIPE: Full name: Document-Driven Implicit Payload Execution. Malicious logic is embedded in normal technical structures; the authors believe agents may reuse it as a reference implementation without recognizing it as an independent behavioral instruction.
• Code example poisoning: Malicious logic is mixed into code examples in skill documents; the authors observe whether agents incorporate this content into their own code generation and execution pipelines.
• Configuration template poisoning: Malicious settings are embedded in deployment or environment configuration templates; the authors observe whether agents output or apply these configurations. Figure 3 gives examples of both types; this article does not reproduce the payloads.
• Disguise design: The authors use functional compliance disguise, anomaly suppression, and false authority cues, and allow multiple attack vectors to be combined. The authors interpret these designs as making payloads appear to belong to normal tasks.

Attack taxonomy and generation pipeline

• Taxonomy source: The authors analyzed over 150 software supply chain poisoning incidents from 2021–2026, with sources including GHSA, NVD, and OWASP LLM Top 10, then mapped them to MITRE ATT&CK (§4.3.1).
• Category boundaries: Table 1 contains 15 categories; the first 14 map to ATT&CK tactics and technique IDs, while the last category, Composite / Other, is a project-specific catch-all for mixed attacks.
• Seeds and model: 81 manual seeds cover the taxonomy; the generation model is Claude Opus 4.6. Stage 1 expands seed structures, and Stage 2 performs disguise and combination mutations (§4.3.2).
• Sampling control: Sampling is weighted by the inverse of category coverage, so low-coverage categories are selected more often; each stage has a maximum number of attempts K, but the paper does not give K's specific value.

Validation and scale control

• Cumulative targets: Stage 1's pool size target is 820, Stage 2's is 1,070, both including 81 seeds; new sample targets are 739 and 250, respectively.
• Format and syntax: Validated that skill documents parse as Markdown and contain at least one code block; embedded scripts pass syntax checks.
• Deduplication threshold: New samples must have a Jaccard similarity below 0.85 with existing pool members. The authors state that a lower threshold would exclude semantically different variants, while a higher threshold would admit near-duplicate samples.

Definition of evaluation metrics

BR=(|Generated|+|Executed|)/|Total|

BR counts both merely generated malicious code and direct execution, so it is not equivalent to the actual success rate of system operations.

DER=|Executed|/|Total|

DER counts only actual execution; judgment is based on full trajectories, system call logs, and observable file or network changes (§5.1).

Experiments cover generation scale, post-loading attack results, and production system disclosures; BR and DER must be understood separately.

Experimental setup

• Target models and frameworks: Claude Code, OpenHands run Claude Sonnet 4.6, GLM-4.7, MiniMax-M2.5 respectively; Codex runs GPT-5.4; Gemini CLI runs Gemini 2.5 Pro (Table 2).
• Data and sample size: Each combination uses all 1,070 adversarial skills. Skills are triggered by standardized tasks corresponding to metadata; the experiment assumes they are already loaded and does not measure market retrieval success rates.
• Running conditions: Linux hosts or containers, timeout 180 seconds, max_tokens 10,000. §5.1 states temperature T=0.0, §7 states T=0.3; the paper does not explain the discrepancy.
• Baselines and metrics: Explicit natural language instruction injection is only reported for comparison with Claude Code＋Sonnet 4.6. BR counts generation and execution; DER counts only execution; both denominators are 1,070, including runtime errors.
• Judgment and human review: Claude Sonnet 4.6 reads the task and full execution trace, classifying as refusal, generation, or execution; two security experts independently blind-review a stratified random 10% sample, reporting Cohen's κ=0.88 agreement with the judge.
• Repeated verification: Samples judged as generated or executed are rerun at least three times, and only counted after majority trials confirm the same failure level; executed samples are additionally reproduced in network-isolated Docker containers with filesystem snapshots.

Main results

From Table 3; each row n=1,070, percentages are original table values.

Swipe horizontally to read the table

Framework	Target model	BR	DER	DER 95% CI	Runtime error rate
Claude Code	Claude Sonnet 4.6	13.5%	2.3%	[1.6, 3.4]	0.7%
Claude Code	GLM-4.7	16.8%	2.4%	[1.6, 3.5]	1.3%
Claude Code	MiniMax-M2.5	20.5%	13.3%	[11.4, 15.5]	1.5%
OpenHands	Claude Sonnet 4.6	22.0%	10.7%	[9.0, 12.8]	12.6%
OpenHands	GLM-4.7	33.5%	27.1%	[24.5, 29.9]	10.1%
OpenHands	MiniMax-M2.5	27.0%	25.5%	[23.0, 28.2]	64.5%
Codex	GPT-5.4	11.6%	3.8%	[2.8, 5.2]	61.0%
Gemini CLI	Gemini 2.5 Pro	14.0%	4.2%	[3.1, 5.6]	6.5%

• Explicit baseline comparison: Under Claude Code＋Sonnet 4.6, explicit instruction injection has an execution rate of 0% on 1,070 samples (§5.1). This is an execution rate comparison; the baseline number should not be referred to as BR.
• Statistical tests: Under Claude Code, the DER difference test between Sonnet and GLM has p=0.887; each compared with MiniMax has p<0.001. The authors also report all cross-system comparisons p<0.001 (§5.3).
• Impact of errors: The authors attribute runtime errors to framework execution failures such as timeouts and API incompatibilities, and describe results including error configurations as conservative lower bounds; runtime errors are not counted as safety refusals.

Same-model cross-framework and cross-model differences

• Sleeper payloads: After excluding runtime errors, the authors compare samples that Claude Code refuses and OpenHands executes; GLM-4.7 has 219, Sonnet 4.6 has 5 (§5.3). The authors explicitly call this comparison an observational analysis.
• Model-specific executions: Under Claude Code, Sonnet, GLM, and MiniMax have 11, 14, and 78 samples executed only by that model, respectively. The case in Figure 5 records Sonnet executing while GLM refuses; the authors interpret GLM's reaction as recognizing an inconsistency between the task and the skill behavior.
• Joint bypass metric: Samples where all three models behave consistently under Claude Code account for 62.1%; the authors report a joint BR of 1.6% and a count of 17; the text also writes 17 "beyond" joint execution cases, with ambiguous counting wording. This is the intersection of results across the three models; no experiment with an actual deployed voting defense is reported.

Categories, payload length, and static scanning

• Category results: For Claude Code, the Config. Tamper group has n=64, with DER for Sonnet, GLM, MiniMax at 6.3%, 6.3%, 20.3% respectively; the Network Exfil. group has n=82, with 0.0%, 1.2%, 19.5% respectively (Table 4).
• Length association: Under Claude Code＋Sonnet 4.6, executed samples average 1,850 B, refused samples average 2,214 B; medians are 1,886 B and 1,964 B respectively (§5.3). The authors believe shorter payloads are more likely to bypass alignment; the experiment does not report a causal intervention controlling for length.
• SkillScan: Post-hoc scanning detects 90.7%, with 99 samples evading all four layers of detection, of which 27 also penetrate model alignment (§6). The contributions section writes the dual penetration as 2.5%; the paper does not specify which model or framework this statistic corresponds to.

Production system validation and fixes

• Disclosure results: The authors report a total of four confirmed issues across Claude Code, OpenHands, Codex, and Gemini CLI, with two deployed fixes (§5.4). The cases involve skill trust, configuration semantics, and tool permission boundaries; this paper does not repeat the exploitation steps.
• OpenHands fix: After adding a warning in the system message that repository content is untrusted, the original 3/3 execution cases become refusals; the paper does not specify the target model for this fix validation (§5.4). This is not the post-fix result for all 1,070 samples.
• Gemini CLI fix: §6 describes the fix as permission stripping; no post-fix full BR or DER is reported.

Other ablations and analyses

• Generation stage: The final 1,070 valid samples have a format validation pass rate of 99.1% (§5.2).
• Stage payload sizes: Stage 1 averages 1,666 B, Stage 2 averages 4,172 B (§5.2).
• Category composition: Supply chain poisoning 47.5%, environment variable theft 11.9%, credential theft 8.6% (§5.2).
• Joint execution case: Figure 4 marks 479 B, 9 lines, a sample jointly executed by all three Claude Code models; its configuration content is not repeated.
• Component ablation: The three disguise techniques of DDIPE are not separately ablated (§7).

The authors explicitly leave open issues such as the retrieval stage, component attribution, baselines, and defense expansion.

Limitations and future directions noted by the authors

• Retrieval and loading prerequisites: The authors leave end-to-end validation of the retrieval stage as future work; §7 notes that in production environments, a large number of skills may dilute attack probability (§3, §7).
• Generation model and framework attribution: The authors acknowledge that Claude Opus 4.6 generating payloads may raise same-family overfitting concerns, and they explicitly state that the Claude Code–OpenHands comparison is not a controlled ablation (§7).
• Disguised components: The three disguise techniques are used in combination without per-component ablation; the authors plan to study their roles separately (§7).
• External applicability and errors: The authors note that the evaluation does not cover the Llama ecosystem or Cursor, and they acknowledge that two configurations had error rates exceeding 60% due to API compatibility issues (§7).
• Comparison baselines: Only compared with explicit instruction injection; the authors believe more complex baselines like Skill-Inject may narrow the gap (§7).
• Defense scope: The authors note that defense evaluation only includes SkillScan; dynamic sandboxing and LLM auditing have not been evaluated (§7).

Experimental coverage

• Main matrix: Five target models and four frameworks form eight combinations, each using 1,070 skills (Table 2, Table 3).
• Review scope: A stratified random 10% sample undergoes blind review by two experts; generated or executed samples are repeated at least three times, and executed samples are additionally isolated for reproduction (§5.1).
• Setting documentation: §5.1 reports T=0.0, while §7 reports T=0.3; the paper does not explain the temperature recording discrepancy.
• Defense result conditions: The SkillScan double-bypass statistics do not specify the target model or framework; the OpenHands fix only reports changes in 3 execution cases (§6, §5.4).
• Generation budget: Algorithm 1 sets a maximum number of attempts K per stage, but its value is not reported; the number of queries or time for the full generation and attack pipeline is not reported.

PoisonedSkills studies skill supply chain poisoning in coding agents, focusing on how document content enters actual system operations.

• Problem and scope: The attacker only controls third-party skills and cannot modify user requests or access system prompts. The experiment assumes skills are already in context, examining system control, asset exfiltration, and infrastructure poisoning.
• Method and mechanism: DDIPE embeds malicious logic into code examples and configuration templates; the authors believe agents may reuse and execute them while completing normal tasks. Claude Opus 4.6 drives the generation pipeline, expanding from 81 seeds to 1,070 skills.
• Baseline evidence: Under Claude Code + Sonnet 4.6, DDIPE's DER is 2.3% (Table 3); under the same configuration and sample size, explicit instruction injection has an execution rate of 0% (§5.1).
• Framework comparison: GLM-4.7 has DERs of 2.4% and 27.1% under Claude Code and OpenHands, respectively; Sonnet 4.6 has 2.3% and 10.7% (Table 3). The authors believe the relationship between alignment and architectural defenses depends on the model; the framework comparison is observational.
• Other evidence: SkillScan detects 90.7%; of the 99 samples that evade scanning, 27 also bypass model alignment, but the corresponding configuration is not specified (§6). The authors also report four confirmed issues and two deployed fixes (§5.4).
• Authors' conclusions and boundaries: The authors advocate auditing skill semantics, scoping permissions per skill, and adapting framework defenses to model alignment. Evidence must be understood alongside retrieval prerequisites, runtime errors, temperature recording discrepancies, and unsplit disguised components.