Secretly Tilting the Scales: Covertly Manipulating Agent Decision Policies via Agent Skills

Core question: can a third-party Skill systematically steer an agent's candidate selection toward an undisclosed target while preserving its declared functionality and output format?

• Problem setting: LLM agents obtain task workflows, tool-use guidance, and output constraints by loading reusable Skills; the authors argue that this makes Skills externalized behavioral policies, and that loading a third-party Skill amounts to delegating part of the decision logic to an external provider, constituting a supply-chain risk.
• Gaps in existing research: the authors state that existing work mainly focuses on execution-oriented or task-boundary threats (malicious code execution, instruction hijacking, unauthorized tool calls, credential abuse); these threats change what the agent does, and do not cover Skills that preserve the authorized task while changing how it decides.
• Core concept: the authors define Skill Policy Integrity, requiring that the behavioral policy induced by a Skill be faithful to its declared functionality and the user's authorized goals; a violation occurs when an undisclosed third-party objective systematically influences behavior while the declared task, available actions, and output interface remain unchanged.
• What the paper proposes: the authors propose SkillShift, a constrained black-box framework that achieves covert policy steering through semantically plausible policy edits, layered validation, failure-guided optimization, and policy compression, instantiated in two scenarios: agent shopping and software dependency selection.
• Threat model:
  • Goal: increase the proportion of target candidates selected while keeping outputs within the candidate set, appropriate to the task, and correctly formatted; the policy should resemble reasonable task guidance rather than explicit prompt injection.
  • Knowledge: black-box—the attacker can observe only final agent responses and cannot access model parameters, probabilities, gradients, hidden states, or attention.
  • Capability: the attacker controls the Skill document and can optimize a shared policy on the development set using only final model outputs; the attacker cannot modify queries, candidates, order, models, or inference settings; query-specific rules, candidate identifiers, positional shortcuts, and explicit target-selection instructions are prohibited, and the target name may appear only in permitted examples.
  • Victim system: LLM agents that load third-party Skills from an open Skill ecosystem; the paper excludes query tampering, candidate poisoning, order manipulation, training-data poisoning, and model modification.

LLM agents and agent safety

• ReAct (Yao et al. 2023) combines reasoning with actions, Toolformer (Schick et al. 2023) lets models call external tools, and AutoGen (Wu et al. 2023) supports multi-agent dialogue; MCP (Anthropic 2024) standardizes the connection of agents to external tools and data.
• Indirect prompt injection embeds malicious instructions in external content processed by the agent (Greshake et al. 2023); InjecAgent and AgentDojo provide evaluation benchmarks for such attacks (Zhan et al. 2024; Debenedetti et al. 2024); StruQ and SecAlign improve instruction–data separation and trusted task alignment (Chen et al. 2025a,b). The paper notes that these works mainly address transient instruction hijacking and unsafe actions, whereas SkillShift manipulates persistent Skill documents to change how legitimate candidates are compared.

Policy and choice manipulation

• Adversarial Search Engine Optimization modifies web pages to favor attacker-controlled results (Nestaas, Debenedetti, and Tramèr 2025).
• In tool-selection settings, MPMA and ToolTweak optimize tool names and descriptions to increase the target's selection rate (Wang et al. 2026b; Sneh et al. 2025), and ToolHijacker injects malicious tool documents (Shi et al. 2026).
• Recommender-system research shows that framing, authority cues and evaluation dimensions can change LLM policy (Filandrianos et al. 2025; Das and Sakib 2024; Deldjoo and di Noia 2024), which the paper says provides a behavioral basis for this attack. The paper notes that the above methods manipulate a single candidate or the task environment, whereas SkillShift studies manipulation on the Skill policy side: the query, candidate set, order and metadata are unchanged, and a shared Skill changes the criteria by which all candidates are compared.

Skill attacks and detection gaps

• Existing work studies malicious Skills in the wild, automated Skill red-teaming and dependency steering (Liu et al. 2026a; Duan et al. 2026; Liu et al. 2026b). The paper notes that Dependency Steering mainly modifies the hard-constraint layer, whereas SkillShift targets the soft policy layer: all candidates remain feasible, but framing, tie-breaking rules and examples change their relative ranking.
• MCPTox focuses on unauthorized actions induced by malicious MCP metadata (Wang et al. 2026a).
• Existing defenses evaluate unsafe agent behavior or detect injected instructions (Ruan et al. 2024; Yuan et al. 2024; Li et al. 2025; Zhu et al. 2025; Hung et al. 2025); the paper notes that they do not explicitly audit persistent selection shifts among legitimate candidates.

Baselines and evaluation targets: the paper compares two non-adversarial controls (No Skill, Clean Skill) and four independently constructed attack baselines (Target Name Only, Keyword Stuffing, Generic Prompt Injection, Direct-Skill Injection, the last being an explicit, non-covert positive control); the detection evaluation covers six tools (skill-scanner-full, Aguara, Snyk Agent Scan, STARS, SkillSpector, ProtectAI DeBERTa) and three registry platforms (ClawHub, Tencent SkillHub, vskill). The authors position the paper as formalizing Skill Policy Integrity and studying covert steering of candidate selection while preserving task effectiveness.

Overall approach: restrict modifications to the Skill to a schema-constrained policy object, attach it to the original Skill with a deterministic renderer, iteratively optimize that object using only the victim model's final responses under the constraint that explicit target instructions are prohibited, and finally compress and freeze it for all evaluations. The method is named SkillShift.

Problem setting and formalization

• Task setting: the agent selects one entity based on the user query q, a fixed candidate set C and the task Skill S; the original document is called Clean Skill, and the appended selection-oriented content is called the attack policy; instantiated as shopping recommendation and Python dependency selection.
• Policy object A = (G, U, T, X), consisting of global evaluation principles G, task-specific rules U, tie-breaking criteria T and semantic examples X; the renderer appends the policy to the base Skill: S_A = R(S_base, A), with the original task instructions, candidate processing flow and output schema unchanged.
• Optimization objective: A^∗ = argmax_{A ∈ Ω} PSR̂_dev(A), subject to the constraint that the valid output rate on the development set is not below the threshold τ_vr and the rendered policy length ℓ(A) does not exceed the budget B; Ω is defined by structural, attack-surface isolation and content constraints.

Structured policy representation and three manipulation mechanisms

• Policy framing: changing the relative salience of task-relevant attributes, i.e., changing the weight of existing evidence without modifying the evidence itself.
• Tie-breaking manipulation: introducing plausible secondary criteria when multiple candidates are equally suitable, changing the relative ranking without making the target an unconditional choice.
• Semantic anchoring: using representative examples to stabilize the application of abstract principles across different queries and backends; examples must not contain query identifiers, candidate position rules or unconditional target mappings.
• The authors' rationale: rewriting the entire Skill without constraints gives a large and hard-to-control search space, may unintentionally change the declared task or output schema, or may achieve high success rates through explicit target instructions; a structured representation makes the allowed policy surface explicit and deterministically verifiable.

Context-consistent initialization

• Starting from an empty policy lacks steering signal, and explicit target-generalization prompts violate the threat model, so the initializer LLM receives the base Skill, the declared task, the specified target and the target's task-relevant attributes, and constructs a small set of plausible evaluation principles, domain rules, tie-breaking rules and examples.
• The initial policy serves both as a legitimate search starting point and as an unoptimized baseline.

Constrained black-box policy search

• In each round, the proposer G generates candidates based on the current best policy, a bounded search history and aggregated feedback; candidates apply local semantic edits (adding or removing rules, rewriting criteria, adjusting priorities, replacing examples, merging overlapping content).
• Layered validation V(A) = Vstr · Viso · Vcon: respectively checks schema and length, ensures only the appended policy is modified, and prohibits explicit target instructions, query-specific mappings, candidate position shortcuts, test-instance references and target mentions outside examples; rejected candidates are not executed, and the violation type is recorded in the history.
• Black-box evaluation: each query in the development set is executed R times, and a deterministic parser E extracts the selected candidate from the final response; invalid or unparseable outputs remain in the denominator and are counted both as invalid outputs and as attack misses; policies are compared lexicographically by higher PSR, higher VR and shorter rendered length; termination occurs after a fixed number of rounds or several consecutive rounds without improvement.

Failure-guided coverage repair and policy compression

• In the repair stage, failed development queries are grouped by requirement and visible candidate attributes, and only category descriptions and aggregate failure statistics are provided to the proposer; a patch is retained only if it passes validation, keeps VR not below τ_vr, improves coverage of the target category and does not substantially reduce overall PSR; the authors say category-level feedback avoids the instance-specific memorization caused by query-level feedback.
• In the compression stage, low-contribution content is removed, semantically similar rules are merged, repeated explanations are shortened and relevant examples are generalized, minimizing policy length subject to a PSR drop of no more than the threshold ϵc and VR meeting the target; compression is validated only on the development set and cannot introduce new target instructions or access held-out results.
• The frozen policy is used for all held-out, cross-model, full-agent and detection evaluations; Algorithm 1 gives the complete procedure.

Experimental setup

• Tasks: shopping recommendation (selecting sports products from a fixed candidate list, target brand Li-Ning) and Python dependency selection (selecting a package from pandas, dask, polars, modin, datatable and returning an import statement, target polars); candidate content, attributes, and order are unchanged between the Clean and Attack conditions.
• Data scale: 50 in-distribution queries per domain, 30 in the development set (dev30) and 20 in the held-out set (heldout20); each query is executed independently 3 times, and dev30, heldout20, and all50 contain 90, 60, and 150 model responses, respectively; policy generation, selection, analysis, and patching use only dev30.
• Models: GPT-5.5 is the Strategy Proposer, and Claude Haiku 4.5 is the main Victim Simulator; the cross-model evaluation backends are Claude Haiku 4.5, DeepSeek v4-Flash, GLM-4.7, Gemini 3-Flash, Qwen3.5-Flash, and GPT-5.4; the full-agent evaluation uses Claude Code (with Claude Haiku 4.5) and Codex-compatible Agent (with DeepSeek v4-Flash).
• Baselines: two non-adversarial controls, No Skill and Clean Skill, and four independently constructed attack baselines: Target Name Only, Keyword Stuffing, Generic Prompt Injection, and Direct-Skill Injection; Direct-Skill Injection is an explicit, non-covert positive control.
• Metrics: PSR (proportion of target candidates selected), VR (proportion that are parseable and selected from the fixed candidate set; the Python task additionally requires the package name to match the import statement), Lift = PSR_atk − PSR_clean (percentage-point difference within the same evaluation batch); invalid outputs remain in the denominator and count as misses.
• Detectors and registries: six detectors (skill-scanner-full, Aguara, Snyk Agent Scan, STARS, SkillSpector, ProtectAI DeBERTa) and three registries (ClawHub, Tencent SkillHub, vskill); downstream code quality uses DeepEval GEval with review model claude-haiku-4-5-20251001, 1 review call per sample, 50 Clean + 50 Attack outputs.

Main results

Independent all50 evaluation (150 responses per condition, per Table 11, corresponding to Figure 2):

Swipe horizontally to read the table

Method	Python PSR	Python VR	Shopping PSR	Shopping VR
No Skill	0.00%	100.00%	33.33%	83.33%
Clean Skill	0.67%	100.00%	37.33%	100.00%
Target Name Only	3.33%	99.33%	46.00%	91.33%
Keyword Stuffing	4.67%	100.00%	76.67%	90.00%
Generic Prompt Injection	32.67%	100.00%	61.33%	74.00%
SkillShift	63.33%	100.00%	81.33%	100.00%
Direct-Skill Injection	100.00%	100.00%	98.00%	98.00%

• The authors state that the steering effect of Target Name Only and Keyword Stuffing is weaker than SkillShift or insufficiently stable; Direct-Skill Injection achieves the highest PSR by explicitly overriding the original selection policy, but it violates SkillShift's covertness constraint and serves only as a positive control.
• Held-out generalization (Table 1): the frozen policy on heldout20 achieves shopping PSR 80.00% (dev30: 85.56%) and Python PSR 75.00% (dev30: 46.67%), with VR maintained at 100.00% in all Attack conditions; the authors caution that the increase on the Python held-out set may reflect split-specific query composition and should not be interpreted as unseen queries being inherently easier.

Cross-model and full-agent transfer

• Cross-model (Table 2, frozen policy, no model-specific optimization): Lift is positive for all model–domain combinations; shopping Lift is 20.00–51.33 percentage points and Python Lift is 52.67–100.00 percentage points; Gemini 3-Flash has the largest Lift in both domains, with Clean PSR 0% and Attack PSR 100.00% on Python, which the authors state should be interpreted as backend-specific saturation rather than universally perfect transfer.
• Full agents (Table 2): on Claude Code, shopping Lift +28.67 and Python Lift +59.34; on Codex-compatible Agent, shopping Lift +28.67 and Python Lift +94.00; Attack VR is at least 96.67%.
• The authors note that model APIs and full agents differ in system prompts, context construction, and Skill injection location, so they focus on the Clean–Attack difference within each backend rather than directly comparing absolute PSR across systems.

Detection, registries, and downstream utility

• Detectors (Table 3, 12): all six detectors detected the SkillShift Attack Skill in 0/6 cases in both domains; ProtectAI DeBERTa's score for the shopping Attack Skill rises from 7.32×10⁻⁵ to 0.3052, but remains below the default threshold of 0.5; Snyk's W007 warning for the Python Clean Skill is unrelated to the attack; Direct-Skill Injection is detected by 4/6 detectors in each domain (66.7%). The authors state that existing defenses have limited coverage of Skill Policy Integrity violations that preserve valid task behavior.
• Registries (Table 14): the SkillShift Skill was accepted or indexed by all three registries without attack-related warnings; handling of the Direct-Skill control was inconsistent—ClawHub flagged it as suspicious, Tencent SkillHub rejected it, and vskill accepted it. The authors state that registry defenses identify explicit instruction override more reliably than plausible policy steering.
• Downstream utility (Table 4): with VR at 100.00% in the selection stage, SkillShift lowers the GEval mean score from 0.558 to 0.506, [email protected] from 28.0% to 12.0% (−16.0 pp), and [email protected] from 66.0% to 46.0% (−20.0 pp); the authors note that the evaluation has only one LLM review call per sample and no executable tests, so this is interpreted as an observed score decrease rather than established functional degradation.

Other ablations and analyses

• Shopping ablation (Figure 3, Table 10, relative to the uncompressed Full Attack at 81.33% PSR): removing target semantic cues −36.00 pp, removing task-adapted scoring criteria −30.00 pp, removing examples −28.00 pp, length-controlled neutral Skill −47.33 pp; removing the specialization framing −1.33 pp and removing tie-breaking rules −2.00 pp, which the authors suggest may reflect empirical fluctuation or component redundancy.
• Python ablation (Table 10, relative to the compressed Lite Strategy at 63.33% PSR): removing all examples −33.33 pp, removing few-shot examples −6.66 pp, removing the line-metadata patch −6.66 pp, and the remaining individual items −1.33 to −5.33 pp.
• Shopping optimization process (Table 7, dev30): initial policy PSR 83.33%, 86.67% after failure repair (+3.34 pp), 85.56% after compression (−1.11 pp), with VR maintained at 100.00%.
• Compression cost (Appendix C): the uncompressed Python policy is 5.34 percentage points higher than the Lite Strategy; the authors state that compression reduces context overhead at the cost of lower attack effectiveness.

Limitations and future directions noted by the authors

• The evaluation is limited to two fixed candidate domains, three runs per query, and a detector case study under a specific configuration (Discussion, Limitations and Future Work).
• The utility analysis relies on automated review (Discussion, Limitations and Future Work); Appendix F further states that the GEval results are a code-quality assessment by a single automated LLM review, not unit-test correctness or an observed execution success rate.
• Future work should cover broader domains, dynamic candidate sets, stronger statistical evaluation, and behavioral defenses (Discussion, Limitations and Future Work).
• Appendix A states that model aliases may point to upstream endpoints that change over time rather than to independently verifiable fixed snapshots, so cross-model results are interpreted through the Clean–Attack difference within each backend; Claude Code exposes only the model family name, and the authors do not claim that the Victim Simulator and Claude Code use the same immutable snapshot.
• Appendix E states that the detection results are a case study of detector behavior, not a comprehensive benchmark of detector accuracy; STARS is evaluated with a rule-based fallback configuration, and the results do not represent the capability of the full STARS system; the shopping STARS results are treated as uncertain because the rules mainly match English.

Experimental coverage

• The subjects under test are two fixed candidate-selection tasks: shopping recommendation (target Li-Ning) and Python dependency selection (target polars), with 50 queries per domain (30 dev + 20 held-out) and 3 runs per query.
• The cross-model evaluation covers 6 backends, the full-agent evaluation covers 2 environments, Claude Code and Codex-compatible Agent; policy optimization uses only Claude Haiku 4.5 as the Victim Simulator.
• The detection evaluation includes 6 detectors and 3 registries; the downstream utility evaluation uses 50 Clean + 50 Attack outputs, with 1 LLM review call per sample.
• The paper does not report statistical significance tests; in the reproducibility checklist the authors state that they did not use statistical tests to judge performance changes (4.12 marked no), did not report a measure of variation or confidence intervals (4.11 marked no), and did not describe the method for setting random seeds (4.7 marked no).
• The paper does not report cost figures such as the number of queries or time consumed by the attack; the optimization budget is given in rounds (30 rounds for shopping search, 20 rounds for patching, 20 rounds for Python search, Table 5).

This is an agent supply-chain security attack paper: the authors treat reusable Skills as externalized behavioral policies, formalize Skill Policy Integrity—the policy induced by a Skill must be faithful to its declared functionality and the user's authorized goals—and study whether a third-party Skill can covertly change candidate selection while preserving the task and output format.

• Method: SkillShift represents the attack policy as a structured object (global principles, task rules, tie-breaking criteria, semantic examples) and manipulates candidate comparison through three mechanisms: policy framing, tie-breaking manipulation, and semantic anchoring; under deterministic constraints such as prohibiting explicit target instructions, it performs black-box iterative optimization using only final model outputs, then freezes the policy after category-level failure repair and policy compression.
• Main results: In the independent all50 evaluation, SkillShift achieves PSR 81.33% on shopping recommendation (Clean: 37.33%) and PSR 63.33% on Python dependency selection (Clean: 0.67%), with VR 100.00% in both (Table 11).
• Generalization and transfer: The frozen policy remains effective on held-out queries (shopping 80.00%, Python 75.00%, Table 1); it transfers to 6 model backends without further optimization (shopping Lift 20.00–51.33 pp, Python Lift 52.67–100.00 pp) and to 2 full agents (Lift 28.67–94.00 pp, Table 2).
• Detectability: The six detectors detected the SkillShift Attack Skill in 0/6 cases, whereas the explicit Direct-Skill Injection control was detected in 4/6; all three registries accepted or indexed the attack Skill (Table 3, 14).
• Downstream impact: With VR at 100%, the GEval code-quality mean score drops from 0.558 to 0.506, and [email protected] drops by 16.0 percentage points (Table 4); the authors therefore argue that effective dependency selection does not guarantee downstream utility.
• Authors' conclusion: The authors state that valid outputs and passing security scans do not guarantee Skill Policy Integrity, and that defenses should introduce Clean–Attack comparisons and behavioral audits with counterfactual tests such as candidate reordering, name substitution, and attribute swapping.