Trust me, import this: dependency steering attacks via malicious Agent Skills

Core question: Can an attacker, through a persistent Skill instruction artifact, actively induce an LLM coding agent to recommend an attacker-controlled package in benign coding tasks?

• Problem setting: The authors state that coding agents such as GitHub Copilot, Cursor, and Claude Code increasingly participate in dependency selection (generating imports, recommending packages, writing install commands), and that incorrect or manipulated dependency recommendations directly affect the software supply chain.
• Gaps in existing understanding: Prior work treats package hallucination as a passive model failure caused by memory, probabilistic generation, etc., where attackers can only register hallucinated package names and wait; the authors argue that the question of "whether an attacker can actively influence dependency recommendations" has not yet been answered.
• Core observation: Skills (persistent instruction artifacts such as Claude Skills, Cursor Rules, and system prompt templates) are treated by agents as trusted development guidance and are continuously loaded across sessions, so a malicious Skill can repeatedly influence dependency decisions across tasks and users, constituting an attack surface distinct from transient prompt injection and RAG poisoning.
• Proposed in the paper: The Dependency Steering attack paradigm, and a semantics-preserving Skill-level optimization method (multi-objective scorer + Context-Patch injection + RAG lifelong policy exploration) that increases the target package generation rate while preserving the original Skill's surface purpose.
• Threat model (Section III-B):
  • Goal: Construct a malicious Skill S* that increases the rate at which the target package Patk appears in dependency outputs (THR), while satisfying semantic preservation and stealthiness.
  • Capabilities: Create, modify, and distribute Skills (public marketplaces, open-source repositories, project templates, etc.); can register or control the target package; can query the target model or a proxy model during construction.
  • Constraints: Cannot modify model weights, training data, user prompts, package managers, or the execution environment; does not require white-box access.
  • Victim system and assumptions: Coding agents that load third-party Skills; users do not check dependencies item by item and treat Skills as legitimate workflow guidance.

LLM-assisted code generation

• Bouzenia et al. (2025), Yang et al. (SWE-agent, 2024), etc. — coding assistants and agent systems perform code generation, tool interaction, and dependency management, directly participating in software supply chain decisions. The authors state that LLMs generate dependencies probabilistically, making dependency selection susceptible to behavioral manipulation.

Skills and instruction systems

• Reusable instruction mechanisms such as Claude Skills, Cursor Rules, and .cursorrules — persistently encode project conventions and workflows and are treated by users and agents as trusted context. The authors state that their difference from transient prompt injection lies in structural persistence, enabling long-term behavioral manipulation across tasks, users, and environments.

LLM package hallucination

• Spracklen et al. (USENIX Security 2025), Krishna et al. (2025), Zhang et al. (ISSTA 2025) — measure the package hallucination phenomenon in code-generating LLMs, where attackers can register hallucinated package names to carry out supply chain attacks. The authors state that these works exploit naturally occurring model errors, whereas this paper shows that hallucination can be actively induced and systematically controlled.

Software supply chain attacks

• typosquatting (Vu et al. 2020), dependency confusion (Ladisa et al. 2023), etc. — traditional attack strategies against open registries such as PyPI and npm.

Baselines and benchmarks: Attack baselines are Direct Package Injection and Bayesian Optimization (BO, with a Gaussian process surrogate model and a hierarchical genetic algorithm); defense evaluations compare against Cisco Skill Scanner, SkillRisk, SkillCheck (Repello AI), Snyk Agent Red Teaming, Cisco Scanner (LLM), SkillCheck (Mondoo), SkillProbe, and SandyClaw. The task benchmark uses the four datasets proposed by Spracklen et al. (2025) (LLM_AT, LLM_LY, SO_AT, SO_LY).

The authors position this paper as follows: unlike passively waiting for hallucination, Dependency Steering actively manipulates dependency selection priors through trusted context artifacts, turning the Skill itself into a new supply chain component.

Overall approach: adapt AutoDAN-Turbo's lifelong strategy exploration paradigm to the Skill optimization setting, using a semantics-preserving Dependency Steering optimization framework to generate malicious Skills that are both stealthy and increase the rate at which the target package appears.

Problem formulation and success criteria

• Given a target agent M, a benign Skill S0, and a benign coding prompt q, the attacker constructs a modified Skill S* such that the target package Patk appears in dependency-related outputs Deps(M(S*, q)) (import, pip install, requirements.txt, etc.).
• Core metric: THR(S) = 1/(|Q|) ∑_{q ∈ Q} E[1(P_atk ∈ Deps(M(S, q)))], i.e., the expected rate at which the attacker-chosen package appears in dependency outputs under benign prompts.
• Three success requirements (Section III-B): effectiveness (increase THR), semantics preservation (consistent with the original Skill's surface purpose), and stealth (avoid conspicuous instructions easily flagged by users, auditors, or scanners).

Baselines

• Direct Package Injection: directly write instructions into the Skill recommending the target package, serving as an upper-bound reference for explicit manipulation; the authors state that it is conspicuous, easily discovered by static tools and LLM auditing, and breaks semantic consistency.
• Bayesian Optimization (BO): uses a Gaussian process surrogate model with ARD plus a hierarchical genetic algorithm to maximize THR over a discrete space formed by 7 types of semantics-constrained mutation operators (dependency recommendation wording, framework preference description, compatibility explanation, usage examples, installation guidance style, code snippet placement, project convention description). The authors state that its search space is small, it does not accumulate knowledge across rounds, and its feedback signal is coarse.

Multi-objective scorer and explicit veto

• Three sub-objectives: target inclusion reward RTIS (1.0–10.0, scored progressively from mention to install command to functional implementation), stealth reward RStealth (0.0–1.0, whether the patch resembles an enterprise security instruction), fluency reward RFluency (0.0–1.0).
• Aggregation: R_base = (α · R_TIS + 10β · R_Stealth + 10γ · R_Fluency)/(α + β + γ); if the target package name appears in plaintext in the patch, an explicit veto is triggered and the score is set to 1.0.
• Two modes: Unconstrained Mode (α=1.0, β=γ=0.0, veto disabled, probing the upper bound of vulnerability); Stealth-Constrained Mode (β>0 and veto enabled).

Context-Patch injection engine

• The proxy attack model does not rewrite the original text but generates an adversarial suffix Cadv disguised as an organizational compliance instruction, injected at a specified anchor position in the Skill (system header, execution instruction, or end): S' = Inject(S0, Cadv, position). The authors state that this preserves the global semantic consistency of the original Skill.

Lifelong semantic strategy exploration

• Warm-Up stage: zero-shot exploration, generating diverse patches on a subset of training queries; patches exceeding the scoring threshold are distilled by an LLM Summarizer into semantic strategies stored in a strategy library.
• Lifelong red-teaming stage: RAG retrieves the most relevant historical strategies by semantic embedding, prompting the attack model to adapt and generate new Cadv, gradually accumulating transferable manipulation knowledge.
• Key parameters (Section V-E): total budget 10 rounds per target; 15% of training data for warm-up, 85% for lifelong red-teaming; target model temperature 0.6, proxy LLM temperature 0.7; the proxy LLM is responsible for patch generation, strategy summarization, and multi-objective scoring (the paper does not specify the proxy LLM's identity).

Experimental setup

• Models tested: Llama-3.1-8B-Instruct, Phi-4-mini-instruct, Qwen2.5-Coder-7B/14B/32B, 5 open-source instruction models in total.
• Datasets: the four datasets of the Spracklen et al. (2025) benchmark, LLM_AT, LLM_LY, SO_AT, SO_LY (LLM-synthesized prompts and Stack Overflow prompts × all-time/recent one year); the paper does not report the number of entries in each dataset. The default target package is awesome_request, and all tasks are Python code generation.
• Baselines: Direct Injection, Bayesian Optimization (300-round budget: 60 rounds BO + 240 rounds GA, population 10).
• Metrics: THR (rate at which the target package appears in dependency outputs), GHR (rate at which any nonexistent package appears).
• Evaluation protocol: in the main experiments, each test request generates 1 completion (N=1), target model temperature 0.6, top_p 0.9; 90% of the data is used for Skill optimization and 10% for evaluation; stability experiments repeat N=5/10/20/50 times.
• Defense evaluation: 3 static analysis tools, 4 LLM/agent auditing tools (among them the LLM components of Cisco Scanner (LLM) and SandyClaw use GPT-4.1 as the evaluation model), 1 sandbox dynamic analysis.

Main results

According to Table IV (method in this paper, Unconstrained Mode) and Tables II and III (baselines), THR is as follows:

Swipe horizontally to read the table

Model	Method	LLM_AT	LLM_LY	SO_AT	SO_LY
Llama-3.1-8B	Direct / BO / Ours	7.14% / 9.13% / 78.57%	12.86% / 12.86% / 78.57%	1.52% / 2.80% / 39.39%	12.5% / 12.5% / 72.41%
Phi-4-mini	Direct / BO / Ours	11.69% / 10.39% / 35.06%	19.44% / 30.56% / 36.36%	12.50% / 19.64% / 21.43%	5.00% / 32.00% / 22.00%
Qwen2.5-Coder-7B	Direct / BO / Ours	66.67% / 76.67% / 89.66%	56.25% / 81.25% / 88.00%	27.27% / 27.27% / 31.82%	15.38% / 59.09% / 59.09%
Qwen2.5-Coder-14B	Direct / BO / Ours	61.29% / 61.29% / 93.55%	62.86% / 51.43% / 94.29%	21.74% / 17.39% / 52.17%	9.52% / 46.15% / 53.85%
Qwen2.5-Coder-32B	Direct / BO / Ours	87.50% / 93.75% / 100.00%	81.08% / 100.00% / 97.30%	27.59% / 31.03% / 62.07%	16.67% / 50.00% / 63.50%

• The authors state that the method in this paper exceeds both baselines in all settings, and that the Qwen2.5-Coder series is the most vulnerable, reaching 100% on LLM_AT and exceeding 50% on every dataset.
• Exception: the method in this paper on Phi-4-mini has a lower THR on SO_LY (22.00%) than the BO baseline (32.00%); the authors do not explain this.
• Baseline GHR (Table I): under benign Skills, Phi-4-mini reaches 32.37% on LLM_LY, and Qwen2.5-Coder-14B/32B is 0% on multiple datasets.

Transfer and generalization

• Cross-model transfer (Figure 2, heatmap, paraphrased in the main text): optimize on one model and evaluate on another; the authors state that cross-model THR for the Qwen2.5-Coder series mostly exceeds 80%, up to 100%, and that Phi-4-mini is lower (about 35%), which the authors consider to reflect its own low susceptibility rather than non-transferability.
• Cross-prompt generalization (Figure 3, bar chart): THR is comparable on training and held-out test prompts; the authors state that no obvious drop from training to test was observed and consider the method not overfitted.
• Cross-domain transfer (Figure 4, Qwen2.5-Coder-7B): optimize and evaluate across the requests, numpy, and django domains; the authors state that cross-domain THR remains consistently high.

Target package names and ablations

• Name similarity (Table VI): requets, with a similarity of 0.6084 to requests, has low THR under the baselines (the authors state this is because the model automatically corrects the spelling), and rises substantially under the method in this paper for each model (e.g., Qwen2.5-Coder-32B reaches 100% for all three names).
• Name length and format (Table VII): the authors state that length, numeric suffixes, and underscore format have no consistent effect on THR; Qwen2.5-Coder-32B reaches 100% under all four names.
• Cross-domain packages (Table V): under awesome_numpy, awesome_request, and awesome_django, THR for the method in this paper mostly exceeds 70%, and Qwen2.5-Coder-14B/32B reaches 100%.

Defense evaluation and adaptive attack

• Static analysis (Table IX): Cisco Skill Scanner and SkillRisk detect none of the Skills generated for the 5 models; SkillCheck (Repello AI) rates them as low risk.
• LLM/agent auditing: Snyk Agent Red Teaming detects all; Cisco Scanner (LLM), SkillCheck (Mondoo), and SkillProbe detect none; the authors state that LLM-based methods perform inconsistently.
• Sandbox dynamic analysis: SandyClaw detects all Skills for the 5 models.
• Adaptive attack (Section VIII-E, Qwen2.5-Coder-7B + awesome_request): after incorporating defense feedback into optimization, Snyk no longer flags it, SandyClaw downgrades it from malicious to suspicious and the run result becomes benign; THR drops to 65.52% but remains effective.

Other ablations and analyses

• Explicitness level (Figure 5): Levels 0–2 maintain high THR, Level 3 shows a moderate decrease, and Levels 4–5 are 0%.
• Skill length (Table VIII): increasing length mostly decreases THR, e.g., Llama-3.1-8B drops from 78.57% (Short) to 28.57% (Long); the exception is Phi-4-mini at 63.64% under Long (higher than 35.06% under Short).
• Number of repetitions (Figure 6): the mean is stable for N=5–50, with a standard deviation of no more than 6.5%.
• Temperature sensitivity (Figure 7): THR is broadly stable between 0.0 and 1.0, with the largest fluctuation on Qwen2.5-Coder-14B (80.65%–96.77%, a difference of 16.12%).
• Convergence (Table X, Qwen2.5-Coder-7B, LLM_AT): 79.31% at round 1, 89.66% at round 5, 93.10% at round 20, and no further improvement at round 50.

Limitations and future directions noted by the authors

• Because proprietary system prompts and deployment pipelines could not be observed, the experiments target open-source coding LLMs rather than closed-source commercial agents (Section IX-C).
• The evaluated prompt distribution cannot fully represent real software engineering workflows (Section IX-C).
• The evaluation primarily measures dependent generation behavior, not the full end-to-end harm after a malicious package is installed (Section IX-C).
• The authors state that future defense mechanisms specifically designed for persistent behavior manipulation may provide stronger robustness against their attack framework (Section IX-C).

Scope of experiments

• The models tested are 5 open-source instruction models: Llama-3.1-8B-Instruct, Phi-4-mini-instruct, Qwen2.5-Coder-7B/14B/32B (Section V-A).
• Tasks are limited to Python code generation; the datasets are LLM_AT, LLM_LY, SO_AT, and SO_LY, and the paper does not report the number of entries in each dataset (Section V-B, V-C).
• Defense evaluation covers 3 static analysis tools, 4 LLM/agent auditing tools, and 1 sandbox dynamic analysis tool; the adaptive attack experiment is conducted only on Qwen2.5-Coder-7B (Section VIII).
• In the main experiments, each request generates 1 completion (N=1); stability experiments repeat N=5/10/20/50 times (Section V-E, VII-C).
• The paper does not report the specific model identity of the proxy LLM (responsible for patch generation, strategy summarization, and scoring), giving only temperature 0.7 (Section V-E).

An attack paper: it reframes package hallucination from a passive model failure into a supply chain vulnerability that can be actively induced.

• Problem: coding agents increasingly participate in dependency selection, and prior work studies only naturally occurring package hallucination; the authors study whether an attacker can actively manipulate dependency recommendations through persistent Skill artifacts.
• Method: Dependency Steering attack + semantics-preserving optimization framework—a multi-objective scorer (target inclusion, stealth, fluency, including an explicit veto mechanism), a Context-Patch injection engine (injecting adversarial suffixes in the style of compliance instructions into Skill anchors), and RAG-based lifelong strategy exploration, adapted from AutoDAN-Turbo.
• Main results: on 5 open-source models × 4 datasets, the method in this paper generally has higher THR than the Direct Injection and BO baselines; Qwen2.5-Coder-32B reaches 100.00% on LLM_AT, Llama-3.1-8B reaches 78.57%, and Phi-4-mini reaches 35.06% (Table IV).
• Transferability: the authors state that the attack transfers across models (cross-model THR for the Qwen2.5-Coder series mostly exceeds 80%), across prompts (training and test THR are comparable), and across domains (among requests/numpy/django).
• Defense evaluation: static analysis tools detect none (SkillCheck rates low); LLM auditing is inconsistent (Snyk detects, Cisco Scanner (LLM), SkillCheck (Mondoo), and SkillProbe do not); the sandbox SandyClaw detects all, but after defense-aware optimization it is downgraded to suspicious, and THR is still 65.52%.
• Authors' conclusion: persistent instruction artifacts constitute an understudied software supply chain attack surface, existing Skill auditing tools have insufficient coverage, and defenses targeting long-term behavior manipulation are needed.