SoK: Agent Skills—Beyond Tool Calling in LLM Agents

Agents have to re-derive execution strategies from scratch every time, and procedural experience disappears when the context window ends.

• Scenario: The authors state that LLM agents can already browse the web, write and debug software, sequentially orchestrate tools, and form multi-agent teams, but new tasks still require re-deriving strategies. Successful procedures do not carry over to the next time.
• Existing gaps: The authors state that existing surveys separately cover agents as a whole, tool use, or multi-agent coordination, and none traces the full lifecycle from acquisition to governance with a skill-centered view.
• Core observation: A skill should be simultaneously executable, reusable, and governable, and carry its own applicability conditions, termination criteria, and callable interface. It is not an atomic tool, a one-off plan, or episodic memory.
• Threat model: §7 treats the skill layer as a new attack surface, not an attack experiment implemented by the authors themselves.
  • Target: A compromised skill can appear harmless in metadata and steer the agent toward malicious outcomes at execution time.
  • Capabilities: Poisoning retrieval metadata, placing unauthorized instructions or code in policies, cross-tenant access through shared libraries, changing behavior through environment changes, misusing privileged skills with untrusted observations, and manipulating applicability-condition inputs.
  • Victims: LLM agents using skill libraries, and skill packages distributed through marketplaces.
  • The six category names are copied verbatim: poisoned skill retrieval, malicious skill payloads, cross-tenant leakage, skill drift exploitation, confused deputy, applicability condition poisoning.
• What this paper does: The authors give a quadruple definition, a seven-stage lifecycle, seven design patterns, and a representation × scope taxonomy, and survey security governance and deterministic evaluation.

The related work is divided by the authors into the procedural memory tradition, agent and tool surveys, adjacent abstractions, and the evaluation comparisons they relay.

Procedural memory and classical planning

• ACT-R and Soar: Anderson (2004) distinguishes declarative memory and procedural productions; Laird (2012) formalizes procedural memory in the Soar cognitive architecture. The authors use these to illustrate that skills encode how to act, and borrow chunking from Chase and Simon (1973) as an analogy for compressing multi-step processes.
• Options framework: An option in Sutton et al. (1999) is (I, π, β). The authors state that this paper's applicability conditions correspond to the initiation set and termination conditions to β, but reinforcement learning options are implicitly selected by a meta-policy and do not address explicit invocation at runtime.
• Planning formalisms: HTN (Nau et al., 2003), BDI (Rao and Georgeff, 1995), and STRIPS/PDDL (Fikes and Nilsson, 1971) already have preconditions, effects, or context conditions. The authors state the main difference is in representation: classical formalisms assume symbolic states, while LLM skills operate on natural language observations.

Agent surveys, tools, and templates

• Survey grouping: The authors state that a set of LLM agent surveys, tool-use surveys, and multi-agent coordination surveys do not adopt a skill-centric perspective to trace the full lifecycle. The text does not elaborate on these surveys' methods one by one here.
• Toolformer: Schick et al. (2023) study LLMs learning to call tools autonomously. The authors state that such behavior usually still stops at a single call; skills can call tools, but add applicability logic, multi-step sequencing, and explicit termination.
• Templates and memory: The prompt templates of White et al. (2023) are static text injected into context. Episodic and semantic memory store observations and facts. The authors state that templates lack applicability conditions, termination logic, and a callable interface; memory stores what happened, not how to act.

Supply chain and prompt injection analogy

• Open-source supply chain: Ladisa et al. (2023) systematize attacks on the open-source software supply chain. The authors use this as an analogy for malicious payloads in code skills, without rewriting that paper's conclusions to claim the skill marketplace already has the same protections.
• Indirect prompt injection: Greshake et al. (2023) study untrusted data hijacking applications with integrated LLMs. The authors use this to illustrate that the instruction-following boundary on which natural language skills rely can be penetrated.

Baselines and benchmarks: The paper itself does not run comparative experiments. It relays the three conditions of SkillsBench, namely no skills, curated skills, and self-generated skills; Table 7 maps SkillsBench, WebArena, Mind2Web, OSWorld, SWE-bench, GAIA, AgentBench, and AndroidWorld.

The authors state that none of these works connects acquisition, execution, evaluation, and governance in a skill-centric way, and this paper adds that perspective with a lifecycle and two taxonomies.

The authors do not propose a new attacker, but define skills with a quadruple and archive literature by lifecycle, design patterns, and representation × scope.

Formalization

A skill is defined as S = (C, π, T, R). C determines applicability under observation and goal, π maps observations and history to atomic actions or skill calls, T decides whether to end relative to the current goal, and R gives name, parameters, and return type.

• Minimality: The authors argue that removing C prevents self-selection, removing T leaves callers unaware of when to resume, removing R prevents programmatic invocation of internal knowledge, and removing π leaves only non-executable metadata.
• Conventions: Implementations often compute a soft applicability score then threshold. In orchestrator architectures, C and T can be external; the quadruple describes the logical interface. π can be a prompt template, code, learned controller, or hybrid.
• Diagram: Fig.1 caption draws observation O as input to applicability gate C, π produces action A, T decides continue or stop, R wraps the module as a callable boundary; goal G can be encoded in O or passed separately.

Literature screening and classification

• Search and inclusion: Six databases, keywords plus forward and backward citation. LLM agent literature window is January 2020 to February 2025, with exception for SkillsBench from February 2026. Of about 180 candidates, 65 retained, 24 systems enter the mapping table.
• Exclusion: Single tool calls, prompt engineering without persistent reuse, multi-agent papers without skill abstraction, and instruction tuning without explicit skill representation.
• Classification process: Record representation, acquisition, execution model, storage, and governance features, revised over three rounds until every deeply analyzed system can be classified. Patterns are deliberately non-exclusive. The authors claim orthogonality of representation and scope because most cells in the result matrix have examples.

Lifecycle

Fig.2 caption: solid lines are main forward path, dashed lines are refinement and retirement feedback. Seven stages: discovery, practice/refinement, distillation, storage, retrieval/composition, execution, evaluation/update.

• First half: Discovery of repeated patterns, failure modes, or bottlenecks; practice via trial-and-error, reflection, and external feedback; distillation condenses trajectories into quadruples with metadata and usage constraints. The authors state that practice improves stability, distillation changes representation.
• Second half: Storage requires indexing, versioning, and metadata; composition checks interfaces, context, and dependencies; execution is constrained by sandbox, permissions, and resources; then drift is monitored, and revision, replacement, or retirement follows.
• Loop: Evaluation can return to practice, retrieval failure to storage, execution failure to discovery. The authors list unsupervised discovery without task definitions or explicit success signals as an open problem, not providing a completed algorithm in the method.

Seven design patterns

Fig.3 caption: patterns arranged along autonomy spectrum, from human-controlled metadata disclosure to autonomous meta-skills; marketplace distribution spans the full spectrum, dashed lines indicate commonly combined patterns.

• P1–P3: P1 registers short description and trigger info first, loads full text only when selected. The authors state this allows registering many skills while consuming context only for activated skills, with risk that metadata errors cause wrong or missed selection. P2 writes policies as executable programs; the authors state same input yields same output, hence testable, but brittle to API, interface, or environment changes. P3 uses hard gating to fix step order; the authors state it is auditable; if rules can be modified by prompt injection, enforcement fails. The authors note P3 operates at the controller level and is not necessarily a reusable skill artifact.
• P4–P6: P4 decides after a task whether to distill new skills or revise old ones; the authors treat quality control as central tension. P5 places natural language instructions and executable fragments in the same package, with conflict resolution needed. P6 creates, modifies, or combines other skills. The authors view Self-Instruct (2023) and CREATOR (2023) as training-time precursors; strict P6 is reserved for runtime-callable generators, and each step needs quality gates to avoid error amplification.
• P7: Skills are versioned packages with dependencies, compatibility, and governance metadata. Scale examples include ToolLLM's over 16,000 real APIs, and ClawHub with over 10,700 published skills within weeks and project stars over 200,000. The authors state community distribution brings supply chain risks. Table 4 compares context cost, determinism, composability, and governance surface; the authors state that quantitative latency-accuracy trade-offs remain an open empirical question.

Representation, acquisition, orchestration, and governance frameworks

• Two axes: Five representations: natural language, code, tool macros, parameterized policies, and hybrid. Six scopes: single tool, multi-tool orchestration, web, desktop, software engineering, robotics/physical. The authors state that web skills encoding high-level intent are more robust to interface changes than skills bound to specific UI elements.
• Acquisition and routing: Five levels: human-written, demonstration distillation, self-practice, curriculum and feedback, meta-skills. Routing can be embedding similarity top-k, or agent reasoning over metadata, or two-stage. When multiple applicability conditions are true, the authors state existing systems mostly use ranking heuristics, lacking explicit conflict resolution strategies. Failure recovery can be made into a skill, whose trust should not be lower than the recovered skill.
• Trust and evaluation: The authors propose four trust tiers. Tier-1 sees only name and description; Tier-2 loads natural language instructions; the authors state isolation only if reasoning and actions are architecturally separated; Tier-3 requires approval or sandbox; Tier-4 operates without per-action approval under preset permissions and monitoring. Tiers should be enforced at runtime. Evaluation dimensions: correctness, robustness, efficiency, generalization, safety. The authors advocate deterministic verifiers comparing environment state against expected outcomes.

The quantitative results are mainly SkillsBench and ClawHavoc material recounted by the authors, not new model experiments run by the authors.

Experimental setup

• Literature corpus: Six databases, January 2020 to February 2025, with the exception of SkillsBench from February 2026 included. About 180 candidate papers, 65 retained, and 24 systems mapped in depth. The corpus includes eight benchmark environments, seven design patterns, and five representations.
• SkillsBench: 86 tasks, 11 domains, 7 agent-model configurations, 7,308 trajectories. Each task has three conditions: no skill, curated skill, and self-generated skill. The judge is a deterministic verifier checking environment state. The paper does not list the names of all 7 configurations, nor does it state the LLM judge model or human agreement.
• Named models: The main text gives only partial numbers for Claude Haiku 4.5, Claude Opus 4.5, Claude Opus 4.6, and Codex + GPT-5.2. Unreported model × condition cells are not filled in.
• Security case: ClawHavoc targets ClawHub for OpenClaw. The numbers come from security research and audits recounted by the paper, not from the authors' own attack experiments.
• Metrics: Pass rate, percentage-point change, malicious skill count, and flaw proportion. The paper does not report statistical significance tests for these differences.
• Pattern co-occurrence: The median number of patterns in the deeply analyzed systems is 2, with a range of 1–4 (§5.9).

Main results

Swipe horizontally to read the table

Reporting unit	Condition	Value	Source
Average across all 7 configurations	No-skill pass rate	24.3%	§8.4
Average across all 7 configurations	Curated-skill pass rate	40.6%	§8.4
Average across all 7 configurations	Curated skills relative to no skill	+16.2 percentage points	§8.4
Average across all 7 configurations	Self-generated relative to no skill	−1.3 percentage points	§8.4
Claude Haiku 4.5	With skills	27.7%	§8.4
Claude Opus 4.5	No skill	22.0%	§8.4
Claude Opus 4.6	Self-generated skills	+1.4 percentage points	§8.4
Codex + GPT-5.2	Self-generated skills	−5.6 percentage points	§8.4

• The authors state that curated skills show a quantifiable improvement, with the effect varying greatly across domains; they also state this is consistent with the hypothesis that domains with less pretraining procedural data benefit more. The authors also write that domain differences could come from task construction, verifier strictness, and skill writing quality.
• The authors state that self-generated skills show no average benefit, with only one of the five tested configurations improving. The positive-change configuration named in the text is Claude Opus 4.6, and the negative-change configuration is Codex + GPT-5.2. The paper does not state whether these five are all of the seven configurations.
• Using the Haiku and Opus pair of numbers in the table, the authors state that a smaller model with curated skills can reach or exceed a larger model without skills. The authors describe this interpretation as an explanatory hypothesis based on benchmark data, not a causal conclusion.

Domain differences and skill forms

• What was measured: Across 11 domains, the main text gives the curated-skill percentage-point change for five domains, along with comparisons of module count and length.
• Results: Medical +51.9, manufacturing +41.9, cybersecurity +23.2, software engineering +4.5, and mathematics +6.0 percentage points (§8.4; §5.10.2 repeats medical, manufacturing, software engineering, and mathematics). Focused skills with 2–3 modules are +18.6 percentage points, skills with 4 or more are +5.9 percentage points; "detailed" skills are +18.8 percentage points, and "comprehensive" documentation is −2.9 percentage points.
• Author interpretation: The authors state this is consistent with P1 loading only focused procedures rather than pouring in full reference material. The percentage points for the remaining six domains are not given in the main text.

ClawHavoc supply-chain case

• What was measured: Malicious skill activity on ClawHub, the OpenClaw skill registry. The authors use it as an empirical anchor for the threat categories in §7.1.
• Results: §7.6 writes 1,184 malicious skills, and the abstract writes nearly 1,200. A Snyk audit states that 36.8% of published skills contain at least one security flaw; another Snyk figure is that 283 of 3,984 skills (7.1%) expose sensitive credentials in plaintext in context windows and output logs. VirusTotal analyzed more than 3,016 skills, and the authors state that hundreds of them show malicious characteristics. Among 12 publisher accounts, a single account corresponds to 677 packages, accounting for 57% of the entire malicious list. The most-downloaded skill contains 9 vulnerabilities, 2 of which are critical, with its ranking inflated by 4,000 fake downloads. The authors state that 91% of malicious skills contain prompt-injection payloads, and more than 135,000 exposed OpenClaw instances are distributed across 82 countries. A reference note separately writes that the cited blog's update reports 824 malicious skills, differing from the 1,184 in the main text.
• Author interpretation: The authors state that this incident falls simultaneously under poisoned retrieval, malicious payloads, confused deputy, and applicability-condition poisoning. P7 is the distribution channel, and P2 is the main execution vector because skills run code with the agent's full system privileges. The authors argue that P3's hard gating makes it less likely to be bypassed by prompt injection alone, and that P4, if it absorbs malicious community skills, would let poisoning enter the generation loop. The authors state that the peak malicious-skill rate is an order of magnitude higher than the point-in-time malware rate of mature registries such as npm; the paper does not give the latter figure.

Negative increments and scan reports

• Negative increments: Of 84 tasks, 16 decline after using skills, with the worst at −39.3 percentage points. The authors state that the worst cases occur when the base model already performs well and the skill introduces conflicting guidance, and they use this to emphasize that applicability conditions should activate only when procedural knowledge is beneficial. The total task count is elsewhere written as 86; the paper does not explain the difference between 84 and 86.
• Scan: The paper recounts a report from the Agent Skills Guard authors: 39 test cases, including 4 adversarial samples flagged as Benign by VirusTotal, with no false positives for legitimate skills in that set. The paper does not separately give the number of legitimate samples, nor does it generalize that result to all skills. The in-text example of reputation scores is that above 80 is considered safe and below 30 malicious.
• Co-occurrence: The most common combination is P1+P7, appearing in 4 systems: HuggingGPT, MetaGPT, AutoGen, and ToolLLM. Claude Code and OpenClaw each use 4 patterns. Five systems use only one, and twelve use exactly two.

Other ablations and analyses

• The paper does not report query counts, token overhead, or end-to-end latency for skill retrieval; the authors state this trade-off remains an open problem (§5.9).
• The authors state that the systems in Table 5 are mostly concentrated in code representation, self-evolving libraries, and game or software-engineering scopes (§5.10.3, §9.1).
• Table 7: The authors state that no single benchmark covers all skill dimensions. The main text does not give each benchmark's scores cell by cell, and this summary does not transcribe cells that could be misplaced.
• The four trust tiers have no corresponding interception rate or false-rejection rate; the authors recommend that untrusted skills default to Tier-1 (§7.2).
• Voyager's self-verification success rate and AgentBench cross-environment results are described by the authors as partial supporting evidence, but they do not directly measure the comparison between curated and self-generated skills (§8.4).

The authors frame the corpus of systematization work, taxonomy validation, and benchmark extrapolation as limitations, and frame validation and governance as open problems.

Limitations and future directions noted by the authors

• Corpus timeliness: Most analyzed systems were published in 2023–2024. The authors believe the LLM skills literature may still be unstable, and the taxonomy may need revision (§9.2).
• Search coverage: The authors state they may still miss industrial systems with limited public documentation, non-English literature, and contemporaneous preprints (§9.2).
• Taxonomy validation: The seven patterns have not undergone external expert surveys or formal concept analysis; non-exclusivity complicates category analysis and may limit discriminative power for new combinations (§9.2).
• Production and security literature: The authors state the corpus emphasizes research systems with published evaluations; LangChain/LangGraph, DSPy, AgentHarm, and InjectAgent are underrepresented in the analysis due to limited peer-reviewed documentation (§9.2).
• Benchmark extrapolation: The authors state the evaluation analysis relies on published benchmarks, which may not reflect longer time horizons, messier environments, or adversarial conditions (§9.2).
• Future directions: §10 lists verifiable autonomous skill generation, unsupervised skill discovery, formal verification across representations, robustness under environmental drift, and governance economics and accountability for marketplaces. §11 again lists discovery, validation, and governance as closing problems.

Experimental coverage

• The literature procedure is in §3: six databases, January 2020 to February 2025, with SkillsBench included as an exception; approximately 180 papers screened down to 65, with 24 systems entering Table 2 and Table 5.
• The taxonomy outputs are seven design patterns, five representations, and six scopes; pattern co-occurrence statistics are in §5.9, with a median of 2 and a range of 1–4.
• The security analysis covers the six threat categories in §7.1, four trust tiers, and the ClawHavoc case, with in-text figures including 1,184 malicious skills, a 36.8% flaw rate, and plaintext credential exposure in 283 of 3,984 skills (7.1%).
• Skill utility figures come from the cited SkillsBench: 86 tasks, 11 domains, 7 agent-model configurations, 7,308 trajectories, and three conditions; §8.4 states these quantitative findings come mainly from this single non-peer-reviewed benchmark.
• The paper does not report the number of repetitions, query budgets, or reviewer–human agreement for experiments the authors ran themselves; the names of the 7 configurations are also not listed individually in the text.

This is a knowledge systematization of the LLM agent skill layer, not a new attack or defense experiment by the authors.

• Problem: Long-horizon workflows need reusable procedural modules. The authors argue that tools, one-off plans, and episodic memory do not simultaneously have applicability conditions, termination criteria, and a callable interface.
• Framework: Skills are written as a quadruple. The literature is archived by seven stages from discovery to update, seven encapsulation and execution patterns, and five representations and six environment scopes. On the security side, six threat categories and four trust tiers are organized; on the evaluation side, correctness, robustness, efficiency, generalization, and safety are organized.
• Utility: The authors recount SkillsBench. Across all 7 configurations, curated skills raise the average pass rate from 24.3% to 40.6%, and self-generated skills change by an average of −1.3 percentage points. In the medical domain, curated skills are +51.9 percentage points, and in software engineering +4.5 percentage points.
• Comparisons and exceptions: Claude Haiku 4.5 with skills is 27.7%, and Claude Opus 4.5 without skills is 22.0%. Skills with 2–3 modules are +18.6 percentage points, and comprehensive documentation is −2.9 percentage points. Of 84 tasks, 16 decline, with the worst at −39.3 percentage points. The authors state these interpretations are not causal conclusions.
• Supply chain: §7.6 records 1,184 malicious skills on ClawHub. Snyk states that 36.8% of published skills have at least one security flaw, and another 7.1% expose credentials in plaintext. The authors state that governance will lag as the skill ecosystem grows.
• Conclusion: The authors argue that whether a skill exists and skill quality should be separated; curated skills and self-generated skills should not be conflated. Discovery, verification, and governance remain open problems.