ACaM: automated capability modeling for agent security
ACaM: Automated Capability Modeling for Agentic AI Security
The authors propose ACaM, which uses static analysis plus an LLM to build capability graphs from agent code, achieving 0.983 macro-F1 on 24 annotated agents and discovering 123 Critical paths in 43 real agents.
ACaM does not trust tool descriptions and assumes an adversary hides dangerous capabilities behind benign descriptions (capability laundering), with sources that may be malicious contributors, compromised third-party tools, or unintentional drift; white-box setting, analyzing Python source code before deployment; does not cover self-modifying code, black-box remote components, or code constructed via obfuscation/reflection/runtime.
- An agent's LLM orchestrator reads only the tool's natural-language description and never inspects the implementation; description-code inconsistency (DCI) allows dangerous capabilities to hide behind benign descriptions (capability laundering), and before deployment no one knows what the agent can actually access, exfiltrate and propagate.
- ACaM uses static analysis to build an Agent Capability Property Graph (ACPG) from source code; the LLM only fills in semantic properties that static analysis cannot determine and cannot overturn static facts; deterministic graph traversal is then used for threat chain reconstruction and structural vulnerability identification.
- On 24 annotated agents ACaM reaches 0.983 macro-F1, higher than SA-only's 0.934 and LLM-only's 0.814; threat chain reconstruction is 0.980, about 8 points higher than the strongest LLM-only baseline; on 43 real agents it finds 123 Critical paths, and 10 findings were validated with PoCs.
- The authors note: only Python is supported; the static engine is conservative, and worst-case modeling and sanitizer handling may produce false positives, while file paths constructed at runtime may produce false negatives. The evaluation covers 43 Python agents and 5 frameworks, and the injection variants are manually constructed.
- Evaluated models
- Sonnet 4.6 (claude-sonnet-4-6)Opus 4.8Claude Haiku 4.5qwen2.5:7b
- Benchmarks
- Discovery Corpus(43 个真实智能体)24-agent annotated subset(10 base + 14 injected variants)
- Metrics
- macro-F1precisionrecallaccuracy7-fact threat chain reconstruction scoreper-fact miss ratecost ($)
Researchers from Pennsylvania State University and IBM Research propose ACaM, a neurosymbolic safety analysis method that derives an agent's capability surface from code rather than tool descriptions before deployment. Targeting the capability whitewashing threat caused by description-code inconsistency (DCI), ACaM compiles agent source code into an Agent Capability Property Graph; code facts established by static analysis cannot be overridden by LLM output, and the LLM only supplements semantic judgments and can reconstruct threat chains spanning multiple tool calls. On 24 manually annotated agents, capability property recovery reaches a macro-F1 of 98.3%, and threat chain reconstruction for injected threats scores 0.980, about 8 points higher than the strongest pure-LLM baseline. On 43 unmodified real-world agents, the paper manually confirms 123 code execution and credential exposure paths, provides proof-of-concept exploits for 10 Critical and High findings, and has reported to the relevant maintainers through coordinated disclosure.
Why it was selectedThe paper proposes a method that derives an agent's capability surface from code rather than tool descriptions, detecting dangerous capabilities not declared in descriptions and cross-tool threat chains before deployment.
Paper analysis
What problem does this paper attempt to solve?
The authors aim to solve the problem that before deployment it is impossible to answer "what can the agent actually do": tool descriptions inconsistent with code (DCI) allow dangerous capabilities to hide behind benign descriptions.
Core problem: an agent's true capability surface (what it can access, exfiltrate, and propagate) is unknown before deployment, because existing methods treat a tool's natural-language description as a proxy for its behavior, and no mechanism guarantees that description and implementation agree.
- Scenario and importance: The LLM orchestrator reads only tool descriptions to decide which tool to call and never inspects tool code; divergence between description and code is called Description-Code Inconsistency (DCI), which enables capability laundering—a tool implementation contains dangerous behavior not declared in its description, silently executed on every normal call (e.g., the get_weather example at the start of the paper leaking credentials).
- Shortcomings of existing methods: The authors state existing defenses mostly monitor, sandbox, or constrain agents at runtime; classic static analysis (Pysa, CodeQL, etc.) has no concept of LLM and tool boundaries; DCI-type work compares description and code tool by tool and does not model the composed system or cross-tool threat chains.
- Core observation: Static analysis and LLMs fail in opposite places—static analysis precisely recovers data flow but does not understand code semantics, while LLMs understand intent but lose intermediate hops on long chains and trust tool descriptions; the division of labor should therefore be asymmetric: static analysis decides everything it can decide, and the LLM fills only the semantic residual.
- Paper proposes: ACaM, a neurosymbolic capability modeling pipeline that compiles source code into an Agent Capability Property Graph (ACPG), supporting two downstream analyses: threat chain reconstruction and structural vulnerability identification.
- Threat model:
- Attacker goal: Hide dangerous capabilities behind benign tool descriptions (capability laundering), with sources that may be malicious contributors, compromised third-party tools, or unintentional drift; ACaM does not distinguish among these three.
- Attacker knowledge/capability: Assumes the adversary "hides capabilities behind benign descriptions rather than behind code constructed to evade static detection."
- Analyst perspective: White-box setting, analyzing Python source code before deployment.
- Out of scope: Code that self-modifies after deployment, native or opaque remote components (black-box), and code that evades analysis via obfuscation/reflection/runtime construction; whether runtime filters resist adaptive attacks is a separate question and out of scope.
What related research exists?
The authors divide related work into six categories according to the five properties in Table 1, and state that no category simultaneously satisfies pre-deployment applicability, modeling LLM orchestration, tool compositionality, code grounding and cross-toolchain reconstruction.
Analysis for non-agent systems
- Pysa, CodeQL, Semgrep, and code property graphs—perform taint tracking and pattern matching on source code; the authors state they are fast, precise, and can run before deployment, but have no concept of LLM and tool boundaries, and modeling the LLM as a taint-in-taint-out stub breaks legitimate flows and misses real problems.
- LLM-assisted static analysis (Iris et al. [28]–[36])—uses LLMs to improve path sensitivity or mine bugs; the authors state they analyze classic software code and do not represent the scheduling layer that selects behavior from descriptions at runtime.
- STRIDE GPT—uses an LLM to enumerate threats directly from system design or descriptions; the authors state it never reads the implementation and inherits everything the description omits.
Runtime structure and execution of agent systems
- AgentArmor et al.—build program/data dependency views from runtime traces, with the LLM reasoning at a high level; the authors state it misses hidden capabilities that never manifest as observable effects (i.e., the DCI case) and never grounds in underlying code facts.
- Runtime defenses such as IFC, CaMeL, FORGE, and sandboxes—the authors state such work is necessary, but many problems could be found earlier, and each mechanism presupposes an input it does not itself produce (IFC needs labels, enforcement needs policies).
Description-Code Inconsistency (DCI)
- [1], [2] measure divergence between MCP tool descriptions and code, finding divergence in 10% to 13% of cases.
- MCP-FSD—detects filesystem capability mismatches in MCP servers, confirmed by running individual tools in a sandbox; the authors state it is dynamic, tool by tool, and filesystem-only, whereas ACaM is static, covers a broader capability set, and can reconstruct cross-tool chains.
Baselines and benchmarks: The RQ1/RQ2 baselines are ACaM-SA (all LLM components removed), Pysa (two configurations: no stubs and with TITO stubs), Semgrep (community rules), and LLM-only baselines (Sonnet 4.6 and Opus 4.8, with 6 prompting strategies for RQ1 and 3 for RQ2 Part 1). The evaluation corpus is the Discovery Corpus of 43 real agents and 24 manually annotated agents (10 base + 14 injected variants).
Authors' positioning: In Table 1, ACaM is the only row checked on all five properties; the authors state DCI-type work is closest to theirs but is tool by tool and does not model the composed system, whereas ACaM treats each tool's capability as a node in the full agent capability graph.
How does the paper address this problem?
ACaM compiles agent source code into an ACPG in five stages: static analysis locks facts, the LLM fills only the semantic residual under guardrails, and deterministic graph traversal is then used for queries.
Overall approach: ACaM uses static analysis to recover most of the capability surface from source code, the LLM handles only the semantic residual that structure cannot express, and no LLM output can overturn facts already established by static analysis; the outputs of both are merged into an ACPG for downstream deterministic queries.
AFG Construction and Taint Seeding (Section 4.1)
- AFG nodes and edges: Components (three roles: orchestrator, tool, helper, plus LLM nodes without source code) and conduits (channels through which data enters and leaves functions, such as shared state, files, network calls, environment variables); edges record calls and data flow. Tools and orchestrators are located via framework registration patterns (e.g., LangChain's @tool, LangGraph's add_node).
- Conduit identification: Deterministic rules identify known library patterns, plus one bounded LLM call (Prompt P1, Appendix A.1) to supplement conduits missed by rules; the rule set is built once per codebase and then applied deterministically, and hallucinated conduits match no call site.
- Taint seeding: Each conduit gets an initial label τ(d) ∈ {PUBLIC, INTERNAL, SECRET}; deterministic rules label according to a fixed trust model (external channels attacker-reachable, credential sources as secret), the LLM may propose its own label, and ACaM takes the higher of the two on the lattice PUBLIC ⊑ INTERNAL ⊑ SECRET, so the LLM can only raise, not lower.
Capability Properties (Section 4.2)
- Core properties: A fixed catalog of 19 properties (Table 2), grouped into data access, external reachability, propagation, and LLM interaction; the fixed catalog makes agents directly comparable.
- Extended properties: Open-ended, discovered per agent, with one all-component LLM call (Prompt P2) proposing cross-component patterns (e.g., multi-hop exfiltration via shared state); a proposal is only a question, and whether it holds is filled in by subsequent analysis.
- Property values: All except output_taint_label take YES/NO/UNKNOWN, initially UNKNOWN; when depending on other components they hold intermediate conditions.
Static Analysis Engine (Section 4.3)
- Dual-track taint: The exfiltration track (whether sensitive data reaches a sink) and the injection track (whether untrusted data reaches memory, shared state, or the LLM context) are tracked separately, because a single lattice would merge PUBLIC with sensitive data and erase provenance.
- Field sensitivity and scheduling: Taint is tracked per field (different keys of the same dictionary can have different sensitivity); components are ordered by the AFG's writer-to-reader edges using Kahn's algorithm, with a single forward propagation and no global fixpoint; when a component writes a higher label than the conduit seed, the conduit is raised.
- Output: Per-component conduit read/write records (including observed labels and writers), subprocess crossings, implicit flow guards, and PUBLIC-origin outputs.
Guardrailed Capability Filling (Section 4.4)
- Structural vs semantic properties: Structural properties are directly determined by facts (e.g., if a SECRET-labeled conduit is read then reads_secret is YES); semantic properties (e.g., capability_escalation) require understanding code meaning and are provided by the LLM.
- CONDITIONAL syntax: Values can be v ::= YES ∣ NO ∣ UNKNOWN ∣ COND(φ) ∣ b ∨ COND(φ) ∣ b ∧ COND(φ), where condition atoms can ask whether some component property is YES, or whether it writes at least some label or some specific value to a conduit; when dependencies are unresolved it is recorded as a condition and left to the IFC Fixpoint to resolve.
- SA → LLM → SA three steps: Static analysis first locks the values that this component's facts can establish; then one LLM call per component (Prompt P3) fills only open values, with locked values shown as final results that the LLM cannot change; finally static analysis re-derives mechanically obtainable values. Each LLM call is local, single-shot, and bounded, confining nondeterminism to the semantic residual.
IFC Fixpoint and ACPG (Section 4.5, 4.6)
- Monotone fixpoint: Values only go from open to determined (UNKNOWN or condition becomes YES/NO), and determined values no longer change, guaranteeing termination and non-contradiction; each round propagates secrets along conduits, sets output labels, and checks open conditions, stopping when a round has no changes, with still-open conditions set to UNKNOWN.
- Post-convergence computation: memory_poisoning_risk is answered by AFG reachability (for indirect injection, starting from PUBLIC conduits read by tools along the injection track to LLM nodes); LLM node properties are derived from the tools it schedules (writes_secret etc. take the union over tools, reads_secret only if some tool returns secrets).
- ACPG and traversal: G = (VC, VD, E, λ, ψ), i.e., the AFG plus the conduits' final taint labels λ and the components' resolved capability templates ψ; downstream queries use deterministic, pattern-guided graph traversal with no LLM involved, implementing nine pattern families (Table 11) covering secret exfiltration, indirect injection, code execution, and more.
Implementation and Success Determination (Section 5, 6.1)
- Implementation: Python 3.11, Scalpel v1.0b0 for AST/CFG; supports anchor identification for LangChain, LangGraph, Google ADK, CrewAI, DSPy; default LLM is claude-sonnet-4-6, temperature 0, structured output.
- Downstream task determination: Threat chain reconstruction decomposes each chain into 7 facts (entry_tool, secret_variable, secret_read_function, injection_point, chain, exfil_boundary, exfil_function), scored as (FULL + 0.5 × PARTIAL)/7; structural vulnerability identification is judged true/false positive by an independent verification agent (Prompt P4, Sonnet 4.6, temperature 0) per each category's criteria, all Critical paths are manually reviewed, and some findings are validated with PoC exploits.
What experiments did the paper conduct?
Three RQs: ACaM macro-F1 0.983 on 24 annotated agents; threat chain reconstruction 0.980, about 8 points higher than the strongest LLM-only baseline; on 43 real agents, 123 Critical paths were found and 10 PoCs were built.
Experimental Setup
- Corpus: The Discovery Corpus contains 43 unmodified real GitHub agents, covering six implementation styles: LangChain, LangGraph, Google ADK, CrewAI, DSPy, and direct LLM SDK, averaging about 6.2 tools, of which 9 are multi-agent systems (Table 4).
- Annotated subset: RQ1, RQ2 Part 1, and RQ3 use 24 agents (10 unmodified base + 14 injected variants); the variants inject three types of exfiltration paths per the OWASP ASI classification: A1 (secret into tool return value), A2 (hidden HTTP POST inside tool), A3 (indirect routing across shared state/subprocess), with tool descriptions left unchanged, constituting capability laundering.
- Ground truth: Manual annotation of 19 core property values (RQ1) and 7 facts per injected chain (RQ2 Part 1); extended properties are open-ended and unannotated.
- Metrics: RQ1 computes per-class precision/recall/F1 over six-value classification (YES/NO/UNKNOWN/PUBLIC/INTERNAL/SECRET) and macro-averages; RQ2 Part 1 uses the 7-fact score (FULL + 0.5 × PARTIAL)/7.
- Baselines: ACaM-SA (all LLM components removed); Pysa (no-stub NS and TITO-stub WS), Semgrep (--config=auto, community rules); LLM-only run on Sonnet 4.6 and Opus 4.8 with 6 prompting strategies (RQ1)/3 (RQ2 Part 1), averaging the best configuration per agent, recorded as LLM-only (best).
- ACaM configuration: Default backbone claude-sonnet-4-6, temperature 0; RQ3 compares Opus 4.8 and Haiku 4.5, with 15 repeated runs.
Main Results
Per Table 5 (24 annotated agents, macro-average):
Swipe horizontally to read the table
System F1 Prec Rec Acc ACaM-SA 0.934 0.947 0.936 0.965 LLM-only (best) 0.814 0.831 0.853 0.881 ACaM 0.983 0.980 0.991 0.992 - Authors' interpretation: ACaM-SA is close to the full pipeline because most core properties can be directly determined by structural facts; its gap is mainly in recall (0.936 vs 0.991), i.e., without the LLM it more often misses existing capabilities; semantic properties have no fixed syntactic pattern and are the main weakness of SA-only.
- Group breakdown: On base agents, removing the LLM loses 3% F1 (0.973 → 0.944); on injected variants it loses 7% (0.993 → 0.923); the authors state the hybrid design's largest gain is precisely on the hardest-to-find capabilities.
- LLM-only gap: Figure 4 shows the gap holds across all 12 configurations, and switching to Opus yields no consistent improvement.
Threat Chain Reconstruction (RQ2 Part 1, Figure 5, Table 13)
- Results: ACaM averages 0.980; LLM-only Sonnet 0.903, Opus 0.888; SA baselines Pysa NS 0.342, Semgrep 0.265, Pysa WS 0.225.
- Exception: Pysa with TITO stubs is instead worse (0.342 → 0.225); the authors state the TITO approximation introduces unanalyzable noise, and LLM boundaries must be explicitly modeled rather than approximated.
- Per-fact misses (Figure 6): LLM-only has a 40% miss rate on the chain fact, ACaM 7%; on the other six facts LLM misses 0%–14%, ACaM 0%–5%. The authors state the LLM can name sources and sinks but loses intermediate hops, whereas ACaM traces chains through the capability graph structure.
Structural Vulnerability Identification (RQ2 Part 2, Table 6, Table 7)
- Scale: On 43 unmodified agents, post-validation confirmed 107 Code Execution/RCE and 16 Secret/Credential Exposure paths (all Critical), plus 1,692 Indirect Prompt Injection, 64 SQL Injection, 15 Access Control/IDOR and other medium/high-severity paths, totaling about 1.8K medium/high-severity; before deduplication the verification agent confirmed 99.4% of High and 98.4% of Medium candidates.
- Manual review and PoC: All 123 Critical paths were manually confirmed to exist in the code; PoCs were developed for 10 Critical/High findings across 9 agents (Table 7), 6 achieving arbitrary code execution (three types: in-process REPL, deserialization, write-then-exec); P1–P4 are detectable with core properties only, P5–P8 require extended properties, P9–P10 only extended properties.
- False positive distribution: Two agents account for 85% of all FPs, which the authors attribute to these two agents' specific overly broad conduit classification rules; the largest single FP source is treating internal file reads as an indirect injection vector (under TM-A mostly developer-controlled files, under TM-B a real path).
- Backbone models and PoC: All exploits succeeded with qwen2.5:7b as backbone, all succeeding on the first attempt except P9; Claude Haiku 4.5 blocked direct attempts, but all ten exploits succeeded under more euphemistic phrasing. The authors state some exploits (e.g., P4, P5 deserialization) do not depend on LLM behavior and cannot be prevented by safety training.
Other Ablations and Analyses
- Run stability (RQ3): Over 15 runs under Sonnet 4.6, only 0.5%–2% of property values varied between runs.
- Backbone substitution (Table 8, 24 agents, mean of 15 runs): Sonnet 4.6 F1 0.983 ($0.769), Opus 4.8 F1 0.982 ($0.863), Haiku 4.5 F1 0.952 ($0.743); the authors state static grounding rather than model tier is the main driver of accuracy.
- Cost (Table 9): ACaM $0.769 and 57 seconds per agent (mean); core properties $0.489, extended properties add $0.280; LLM-only averages $0.424 and 112 seconds; ACaM-SA $0 and 5 seconds.
- LLM-only finding real vulnerabilities: The authors report it can locate plausible sources and sinks but cannot reliably reconstruct end-to-end chains, and tends to stop at the first path found.
What are some points that can be further explored?
The authors list limitations including the Python scope and the static engine's conservativeness, and propose several future applications of ACPG; the evaluation covers 43 Python agents and manually injected variants.
Limitations and Future Directions Noted by the Authors
- Python scope (Section 7): ACaM analyzes only Python source code; the AST frontend, framework anchor detectors, and library-call conduit rules are Python-specific, and porting to languages such as TypeScript is near-term future work.
- Static engine conservativeness (Section 7): The engine models the capability surface under worst-case assumptions—once any tool's output contaminates the LLM context, every tool reachable by the LLM is treated as a possible reader; sanitizers are only recorded and not used to lower taint labels; these choices increase false positives (the authors state few appeared in the evaluation).
- Runtime-constructed file paths (Section 7): When paths are concatenated at runtime the engine cannot determine the filename, and such reads are labeled INTERNAL by default, which may cause false negatives.
- Future applications of ACPG (Section 7): Least-privilege policy synthesis, providing policy input to runtime enforcement mechanisms, pairing with tool descriptions for finer-grained inconsistency checks, and overlaying multiple agents' ACPGs to model distributed multi-agent systems.
- Indirect prompt injection (Section 6.3.2): The authors state prompts have no boundary between data and instructions, Indirect Prompt Injection "remains open by construction", solving it is beyond this paper's scope, and ACaM only detects the exposure surface, leaving sanitization choices to developers.
Experimental Coverage
- The subjects are 43 real Python agents (6 implementation styles, 5 frameworks plus direct SDK, 9 multi-agent systems), and RQ1/RQ3 use 24 of them that are manually annotated.
- The injected variants are 14 manually constructed ones, covering the OWASP ASI's A1/A2/A3 three exfiltration path types; the authors state these variants reference disclosed precedents (Bing Chat and M365 Copilot cases and a LangChain CVE).
- Validation: All Critical paths were manually reviewed (123), and 10 findings were validated with PoC exploits (local deployment, synthetic data); High/Medium candidates were judged by a Sonnet 4.6 verification agent, and the paper does not report agreement between that verification agent's judgments and manual review.
- LLM components: Backbone evaluation covers three tiers, Sonnet 4.6, Opus 4.8, and Haiku 4.5; PoCs used two backbones, qwen2.5:7b and Claude Haiku 4.5, without upgrading to Sonnet or Opus (the authors state the pattern is architectural rather than model-specific).
- Repetitions: RQ3 uses 15 runs per configuration; the paper does not state the number of repetitions for the RQ1/RQ2 main results.
Summarize the main content of the paper
ACaM uses a neuro-symbolic approach to build agent capability graphs from code, addressing the problem of tool descriptions masking dangerous capabilities; it reaches 0.983 macro-F1 on the annotated set and discovers 123 Critical paths in real agents.
A paper on pre-deployment static analysis for agent systems, targeting capability laundering caused by Description-Code Inconsistency (DCI): a tool implementation contains dangerous capabilities not declared in its description, the LLM orchestrator sees only the description, and a normal call triggers them.
- Problem: Existing defenses are mostly at runtime; classic static analysis has no concept of LLM/tool boundaries; DCI-type detection works tool by tool and cannot see threat chains composed across tools.
- Method: ACaM has five stages—build an Agent Flow Graph and seed taint, instantiate 19 core properties plus LLM-discovered extended properties, a static analysis engine performs dual-track (exfiltration/injection) field-sensitive taint propagation, guardrailed filling (SA → LLM → SA, the LLM cannot overturn static facts), and an IFC Fixpoint monotonically resolves cross-component conditions, producing an Agent Capability Property Graph (ACPG); downstream, deterministic pattern traversal is used for threat chain reconstruction and structural vulnerability identification.
- Result one (RQ1): On 24 manually annotated agents, macro-F1 0.983, higher than ACaM-SA's 0.934 and the best LLM-only configuration's 0.814 (Table 5).
- Result two (RQ2 Part 1): On 14 injected variants, threat chain reconstruction averages 0.980, about 8 points higher than the strongest LLM-only baseline (Sonnet 0.903), and SA baselines all at or below 0.342 (Figure 5); the LLM misses 40% on the chain fact, ACaM 7%.
- Result three (RQ2 Part 2): On 43 unmodified real agents, 123 Critical paths were confirmed (107 code execution/RCE, 16 credential exposure) and about 1.8K medium/high-severity paths; 10 findings were validated with PoC exploits, 6 of them arbitrary code execution, among which the deserialization RCE requires no LLM decision.
- Result four (RQ3): Only 0.5%–2% of property values vary across 15 runs; with Haiku 4.5 as backbone F1 is 0.952, still higher than the best LLM-only 0.814, and the authors state static grounding is the main driver of accuracy; cost is $0.769 and 57 seconds per agent.
- Authors' conclusion: Static analysis provides structural precision and the LLM provides semantic interpretation, and the hybrid design exceeds either component alone; the authors argue many problems only caught at runtime could have been found before deployment, and ACPG can also support future applications such as least-privilege policy synthesis. Limitations include Python-only support, possible false positives from the engine's conservative modeling, and possible misses on runtime-constructed paths.