Skip to content

We Hold All the Cards: Securing Agent Cards at the runtime trust boundary of the Agent2Agent protocol

We Have All the Cards: Securing Agent Cards at the Operational Trust Boundary of the Agent2Agent Protocol

arXiv: 其他 AI 安全论文· arXiv:2611.18887· Miguel A. Guirao Aguilera, Satyajayant Misra, Efren Lopez-MoralesNew Mexico State University; The University of Texas at DallasIndexed by this site · 2 days ago· Originally publishedSelected Defenses· Its place in the scan record →SelectedAttention40

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose a taxonomy of ten classes of A2A Agent Card manipulation attacks and a layered verification framework, SEC CARD; on real-card injection attacks, the strongest judge achieves 97.0% recall and 1.7% false positives.

Threat modelThe attacker is a malicious provider or has compromised a legitimate provider, controls all free-text fields of the Agent Card, and can publish cards that pass A2A schema validation; the orchestrator and its execution environment are trusted, the Agent Card is treated as untrusted input, and the attack occurs before the provider is invoked (pre-invocation).

Questions
In the A2A protocol, the Agent Card contains attacker-controllable natural-language fields that influence the orchestrator's agent selection before the provider is invoked, constituting a pre-invocation trust problem; the A2A specification provides no guidance on assessing card trustworthiness beyond schema conformance.
Methods
The authors propose a taxonomy of ten categories of Agent Card manipulation attacks and design the layered validation framework SEC CARD: Layer 1 performs protocol structural validation, Layer 2A uses deterministic NLP rules, and Layer 2B uses a read-only, tool-free LLM-as-a-judge for semantic judgment, outputting BENIGN/SUSPICIOUS/MALICIOUS.
Experiments and results
The authors probed 36.1M domains to collect 310 deployed Agent Cards, and built a 660-card real-card perturbation benchmark and a 260-card synthetic adversarial benchmark. The strongest judge, GPT-5.5, achieved 97.0% security recall and 1.7% FPR on the perturbation benchmark; the nine judges achieved 25.8%–92.3% recall on the synthetic benchmark, and the authors state that performance depends heavily on the choice of judge model.
Limitations and future work
Limitations noted by the authors: only pre-invocation card content detection is evaluated; provider runtime behavior is not verified to match claimed capabilities; provenance and integrity mechanisms such as signatures are not replaced. The experiments measure only detection, not end-to-end attack success rates against deployed orchestrators; among wild cards, there are only 28 Untrusted samples.
Experimental setupGPT-5.5、Claude Opus 4.8 et al. · In-the-Wild Dataset(310 张部署中的 Agent Card)、Real-Card Perturbation Dataset(660 张) et al. · security recall、false-positive rate (FPR) et al.
Evaluated models
GPT-5.5Claude Opus 4.8GPT-5.4-miniGemini 3.1 Flash LiteClaude Haiku 4.5Gemini 2.5 Flash LiteLlama 3.1 8BPhi-4 14BMistral Nemo 12B
Benchmarks
In-the-Wild Dataset(310 张部署中的 Agent Card)Real-Card Perturbation Dataset(660 张)Synthetic Adversarial Dataset(260 张)
Metrics
security recallfalse-positive rate (FPR)precisionaccuracyCohen's κ
AI summaryand selection rationaleThe researchers propose a taxonomy of ten classes of discovery-phase manipulation attacks against the Agent Card of the Agent2Agent (A2A) protocol, along with a pre-invocation validation framework, SecCARD. The Agent Card is JSON metadata that an orchestrator reads before invoking an external agent; its natural language fields can be controlled by attackers and influence agent selection. The paper groups the attacks into three categories: manipulating selection, misreporting capabilities, and evading detection. SecCARD combines protocol compliance checks, deterministic NLP rules, and an LLM semantic judge for layered detection. The team probed 36.1 million internet domains and collected 310 deployed Agent Cards; two annotators had 96.8% agreement, and the annotations comprised 282 benign, 27 suspicious, and 1 malicious card. On a perturbation benchmark of 660 real cards injected with controlled attacks, the strongest judge, GPT-5.5, achieved 97.0% safety recall and 1.7% false positive rate, while nine LLM judges had safety recall between 25.8% and 92.3% on a 260-card synthetic adversarial benchmark, so detection effectiveness depends heavily on the underlying model choice.

The researchers propose a taxonomy of ten classes of discovery-phase manipulation attacks against the Agent Card of the Agent2Agent (A2A) protocol, along with a pre-invocation validation framework, SecCARD. The Agent Card is JSON metadata that an orchestrator reads before invoking an external agent; its natural language fields can be controlled by attackers and influence agent selection. The paper groups the attacks into three categories: manipulating selection, misreporting capabilities, and evading detection. SecCARD combines protocol compliance checks, deterministic NLP rules, and an LLM semantic judge for layered detection. The team probed 36.1 million internet domains and collected 310 deployed Agent Cards; two annotators had 96.8% agreement, and the annotations comprised 282 benign, 27 suspicious, and 1 malicious card. On a perturbation benchmark of 660 real cards injected with controlled attacks, the strongest judge, GPT-5.5, achieved 97.0% safety recall and 1.7% false positive rate, while nine LLM judges had safety recall between 25.8% and 92.3% on a 260-card synthetic adversarial benchmark, so detection effectiveness depends heavily on the underlying model choice.

Why it was selectedThe paper presents a taxonomy of ten types of Agent Card manipulation attacks and the SecCARD pre-invocation validation framework; actual tests across nine LLM judges show that detection performance varies greatly.

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    The natural-language fields of A2A Agent Cards can be controlled by attackers to manipulate the orchestrator's agent selection before the provider is invoked; the paper proposes a taxonomy and a pre-invocation validation framework.

    Core problem: In the A2A protocol, attacker-controllable natural-language fields in the Agent Card influence the orchestrator's agent discovery and selection before the provider is invoked, constituting a pre-invocation trust problem, and the A2A specification provides no normative guidance on assessing card trustworthiness beyond schema conformance.

    • Scenario and importance: The orchestrator retrieves the card via /.well-known/agent-card.json, places free-text fields into the LLM context, and then autonomously decides whether to invoke, without human review; the authors state that successful manipulation could lead to selecting an attacker-controlled provider, delegating sensitive tasks, leaking task context, and other consequences.
    • Gaps in existing work: The authors state that existing work characterizes only limited discovery-time manipulation (e.g., A2ASecBench covers only 2 classes of Agent Card attacks), is mainly evaluated in controlled environments, and lacks empirical measurement of cards in deployment, human-annotated security ground truth, and dedicated pre-invocation verification methods.
    • Core observation: The authors divide ten attack classes by detection method into semantic types (requiring inference of intent, e.g., Ranking Manipulation) and structural types (leaving lexical/structural traces, e.g., Unicode Tricks), and argue that a single detection strategy is insufficient and that deterministic and semantic analysis are complementary (Table 3).
    • Paper outputs: A taxonomy of ten classes of Agent Card manipulation attacks; a layered verification framework SEC CARD; three datasets (310 wild cards, a 660-card real-card perturbation benchmark, and a 260-card synthetic adversarial benchmark).
    • Threat model (Sec. 4):
      • Goal: Manipulate the discovery process so that an adversarial provider crosses the discovery-to-invocation trust boundary and enters the autonomous workflow.
      • Knowledge: The attacker needs no prior communication with the target provider and no control over its system prompt.
      • Capabilities: A malicious provider publishes a malicious card from scratch, or compromises a legitimate domain to replace the card; controls all free-text fields of the card, can publish cards that pass A2A schema validation, and can combine the ten attack classes.
      • Victim system and assumptions: The A2A orchestrator (client agent); the paper assumes the orchestrator and its execution environment are trusted, and treats the Agent Card as untrusted input even when it comes from a seemingly legitimate provider.
  2. What related research exists?

    Related work falls into two groups: A2A/Agent Card security and prompt injection; the closest, A2ASecBench, covers only 2 classes of card attacks and has no pre-invocation defense.

    A2A and Agent Card security

    • Habler et al. [12]—a relatively broad threat analysis of A2A, covering risks in the Agent Card discovery stage; Table 1 notes that it does not cover attack categories, wild cards, or pre-invocation defenses.
    • Louck et al. [13]—focuses on privacy and secure information exchange in A2A; He et al. [14] and Sharma et al. [15] study adversarial behavior and safety evaluation of broader multi-agent systems rather than specifically the A2A discovery stage.
    • Li et al.'s A2ASecBench [16]—a security benchmark covering the A2A lifecycle, in which two of six attack categories target Agent Card admission (AgentCard Spoofing, Capability Cloaking); the paper notes that it does not introduce a dedicated pre-invocation Agent Card validation mechanism. This paper's Real-Card Perturbation Dataset adopts the seed-and-perturb construction method it proposes.
    • Deshmukh et al. [17]—an empirical study of Agent Card poisoning, stating that manipulated cards can redirect the orchestrator to attacker-controlled endpoints, and evaluating detection strategies in a controlled testbed; the paper states that this work considers broader discovery-time manipulation and supplements it with internet-scale measurement and human ground truth.

    Prompt injection in LLM applications and agents

    • Perez and Ribeiro [18]—propose prompt injection: instructions embedded in attacker-controllable text influence LLM behavior.
    • Greshake et al. [8]—extend the threat to indirect prompt injection, where malicious instructions come from externally retrieved content; subsequent work [19], [20] demonstrates related risks in tool-integrated agents and external resources.
    • AgentDojo [7], InjecAgent [10], Liu et al. [9]—the paper cites them in the introduction to show that indirect prompt injection is an established threat to LLM-integrated systems, without further elaboration in the main text.

    Positioning: The authors state that this paper studies a different point in the agent lifecycle—untrusted content embedded in discovery metadata—focusing on discovery-time detection; compared with the works in Table 1, this paper simultaneously covers ten attack categories, wild card measurement, human ground truth, and pre-invocation defenses.

  3. How does the paper address this problem?

    First establish a taxonomy of ten attack categories, then design SEC CARD accordingly: Layer 1 protocol validation, Layer 2A deterministic rules, Layer 2B read-only LLM semantic review, outputting a three-level verdict.

    Overall approach: first use a taxonomy of ten attack categories to characterize the discovery-time attack surface, then, based on the observation that "structural attacks use deterministic rules and semantic attacks use LLM reasoning," design the layered validation framework SEC CARD (Secure Card Analysis for Risk Detection), completing validation before card content influences agent selection or invocation.

    Attack taxonomy (Sec. 5)

    • Manipulate Selection: Indirect Injection (embedding natural-language instructions to override orchestrator reasoning), Ranking Manipulation (biasing ranking with exaggerated claims and comparative language), Evaluator Manipulation (requesting a benign verdict from security validators or LLM reviewers).
    • Misrepresent Capabilities: Skill Deception (advertising misleading skills), Skill Chaining (skills that are individually benign but expand privileges when combined), Out-of-Context Skills (skills semantically inconsistent with the claimed purpose).
    • Evade Detection: Unicode Tricks, Context Pollution, URL Pivot, Metadata Gaps, which exploit obfuscated characters, diluted content, endpoint manipulation, and metadata flaws, respectively, to bypass validation.
    • Detection mapping (Table 3): five semantic attack categories are primarily detected by Layer 2B, four structural attack categories primarily by Layer 2A, and Out-of-Context Skills is Mixed (2A+2B).

    Layer 1: Structural compliance validation

    • Function: validates JSON against the A2A v1.0.0 schema; discards cards missing required fields or with malformed structure, filtering out non-agent-card responses (such as generic API error pages).
    • Implementation: the authors' lightweight validator checks whether the eight protocol-mandated fields are all present.

    Layer 2A: Deterministic NLP rules

    • Preprocessing: recursively extracts card text and normalizes Unicode homoglyphs to canonical Latin equivalents.
    • Seven feature groups: instruction patterns, ranking indicators, sensitive-operation indicators, URL analysis, Unicode analysis, context-length analysis, skill-consistency analysis (TF–IDF cosine similarity between agent description and skills).
    • Severity aggregation: feature scores are summed; ≥8 is judged MALICIOUS, 3–7 is judged SUSPICIOUS, and the user prompt is inserted within the <3 判 BENIGN;作者称该层输出确定、无需外部推理 API、本身不受提示词注入影响。

    Layer 2B:LLM-as-a-Judge 语义分析

    • 关键设计:使用独立、只读的 LLM 实例做分类——无工具访问、无记忆、无对话连续性;作者称这样即使评审被注入也不会影响 orchestrator,且注入本身就是恶意证据。
    • 输入准备:只分析影响 orchestrator 决策的字段子集(name、description、provider、url、documentationUrl 及全部 skills 条目),序列化为 JSON 并截断到 6 KB,作者称该上限保留了野生语料 310 张卡中超过 95% 的完整内容。
    • 信任边界:序列化卡片包在专用 <UNTRUSTED_CARD> delimiters, explicitly separated from the trusted system prompt.

    Output and admission

    • Three-level verdict: BENIGN / SUSPICIOUS / MALICIOUS, serving as a security signal for the orchestrator's admission decision; the paper does not prescribe a unified signal-combination policy, stating that it depends on the deployer's tolerance for false positives and false negatives.
    • Success criterion (Sec. 8.2): for attacks labeled SUSPICIOUS, a verdict of either SUSPICIOUS or MALICIOUS counts as detection; for attacks labeled MALICIOUS, only a verdict of MALICIOUS counts as detection (severity-aware security recall).
  4. What experiments did the paper conduct?

    Centered on four RQs: alignment with human annotations on 310 wild cards, attack detection and false positives on a 660-card perturbation benchmark, per-category performance on a 260-card synthetic benchmark, and a horizontal comparison of nine judges.

    Experimental setup

    • Subject under test: Layer 2B semantic judging of SEC CARD, with nine LLM judges in total: GPT-5.5, Claude Opus 4.8, GPT-5.4-mini, Gemini 3.1 Flash Lite, Claude Haiku 4.5, Gemini 2.5 Flash Lite, Llama 3.1 8B, Phi-4 14B, Mistral Nemo 12B; the open-source models were run locally via Ollama on an NVIDIA RTX A2000 12 GB GPU.
    • Datasets: In-the-Wild (310 deployed cards, human-annotated as 282 BENIGN / 27 SUSPICIOUS / 1 MALICIOUS); Real-Card Perturbation (60 benign seeds + 600 adversarial variants, 60 per attack class, 300 each SUSPICIOUS/MALICIOUS); Synthetic Adversarial (260 fully generated adversarial cards, generated by Claude Sonnet 4.6 and human-verified for schema conformance, covering ten attack classes, three obfuscation levels, three attacker sophistication levels, and four application personas).
    • Metrics: security recall (severity-aware: MALICIOUS must be judged MALICIOUS; SUSPICIOUS may be judged SUSPICIOUS or MALICIOUS), FPR, precision, accuracy, three Cohen's κ, average latency.
    • Judging procedure: For RQ1, the labels of two human annotators served as ground truth (agreement 96.8%, κ=0.804, disagreements resolved through discussion); all judges used the same semantic-validation prompt and evaluation pipeline.
    • Number of repetitions: Not reported in the paper.

    Main results

    According to Table 7 (660-card Real-Card Perturbation benchmark, Overall recall and FPR on 60 benign controls) and Table 8 (260-card Synthetic Adversarial benchmark Overall recall):

    Swipe horizontally to read the table

    Judge modelPerturbation benchmark recallPerturbation benchmark FPRSynthetic benchmark recall
    GPT-5.597.0%1.7%92.3%
    Claude Opus 4.891.0%5.0%81.5%
    GPT-5.4-mini86.3%15.0%73.8%
    Gemini 3.1 Flash Lite92.3%6.7%86.5%
    Claude Haiku 4.588.0%15.0%81.9%
    Gemini 2.5 Flash Lite68.8%0.0%70.8%
    Llama 3.1 8B64.2%26.7%63.5%
    Phi-4 14B56.8%26.7%42.7%
    Mistral Nemo 12B33.7%5.0%25.8%
    • The authors state that the strongest models can detect most controlled attacks while retaining most benign seed cards, and that the injected attack content (rather than the underlying real card) can often be identified as the source of the trust violation.
    • The authors state that detection performance varies widely across attack classes and judge models, and that semantic judging is an important deployment choice rather than an interchangeable component.
    • Exact three-class accuracy is consistently lower than severity-aware recall (e.g., GPT-5.5 on the synthetic benchmark: 92.3% recall vs. 77.3% accuracy).

    RQ1: Agreement with human annotation on wild cards (Table 6)

    • What was measured: 310 In-the-Wild cards, with SUSPICIOUS∪MALICIOUS merged into Untrusted (n=28), evaluating only Layer 2B.
    • Results: GPT-5.5 accuracy 94.5%, recall 89.3%, FPR 5.0%, κ=0.718; Gemini 2.5 Flash Lite recall 82.1% but lowest FPR 0.7%, precision 92.0%; Claude Haiku 4.5 and Phi-4 detected all 28 Untrusted cards, but with FPR of 45.7% and 46.5%, respectively.
    • Authors' interpretation: Semantic verification is feasible on naturally occurring cards, but model choice affects the operational trade-off between attack detection and rejection of benign cards; high recall alone does not represent an effective trust decision.

    RQ3: Per-class performance on the synthetic benchmark (Table 8)

    • What was measured: Recall differences of 260 synthetic cards across ten attack classes (n=35 per semantic attack class, n=25 for Mixed, n=15 for structural attacks).
    • Results: GPT-5.5 ranged from 67% on Metadata Gaps to 100% on Indirect Injection, Skill Chaining, URL Pivot, and Unicode Tricks; Ranking Manipulation ranged from 86% for GPT-5.5 to 17% for Phi-4 and Mistral Nemo; Metadata Gaps ranged from 93% for Gemini 3.1 Flash Lite and Claude Haiku 4.5 to 13% for Mistral Nemo.
    • Authors' interpretation: No attack class is consistently difficult across all judges; robustness to one class of manipulation does not imply robustness to the entire discovery-time attack surface.

    Other ablations and analyses

    • Latency (Tables 7, 8): Average per-card latency ranged from 1.1s for Gemini 2.5 Flash Lite to 16.9s for Phi-4 (synthetic benchmark; on the perturbation benchmark, Phi-4 was 15.9s).
    • Severity stratification (Table 7): GPT-5.5 recall was 98.7% on MALICIOUS variants and 95.3% on SUSPICIOUS variants; Phi-4 showed the opposite (41.0% vs. 72.7%).
    • RQ1 evaluating Layer 2B alone: The paper states that Layers 1 and 2A are deterministic components, and RQ1 focuses on the semantic judgment most comparable to human annotation; it does not report end-to-end numbers for the full pipeline on wild cards.
    • Negative results: Small open-source models were generally weak, with Mistral Nemo achieving only 33.7% recall on the perturbation benchmark and 25.8% on the synthetic benchmark; the authors provide no explanation for the cause.
  5. What are some points that can be further explored?

    The authors list two types of limitations—runtime behavior and provenance authentication—as well as future directions such as multi-agent collaboration; the experiments cover only the detection stage, and there are only 28 wild Untrusted samples.

    Limitations and future directions noted by the authors

    • Runtime behavior (Sec. 9.2): the framework evaluates only pre-invocation Agent Card content, measuring detection of adversarial content rather than end-to-end attack success rate against a deployed orchestrator; a provider may have a benign card but malicious runtime behavior, and the authors state that combining post-invocation behavior monitoring with the framework is a future direction.
    • Authentication and provenance (Sec. 9.2): SEC CARD does not replace cryptographic provenance and integrity mechanisms such as JWS signatures; a validly signed card may still contain deceptive metadata, and the two are complementary.
    • Future work (Sec. 10): the authors state that the threat model should be extended to collaborative multi-agent attacks, adaptive attackers who modify cards over time, and attacks on discovery infrastructure such as registries, caching services, and trust management mechanisms.
    • Generalization discussion (Sec. 9.1): the authors state that eight of the ten categories can be directly transferred to MCP, that url_pivot can only be partially mapped, and that metadata_gaps requires adaptation to the MCP schema; applying it to other protocols requires adapting structural validation and detection features.

    Scope of experimental coverage

    • The subjects under test are nine LLM judges (six closed-source, three open-source 8B–14B), evaluating the Layer 2B semantic review; Layer 1 and Layer 2A are not separately reported with end-to-end detection numbers (Sec. 8.1).
    • The three datasets total 1230 cards: 310 wild, 660 perturbed (600 adversarial + 60 benign controls), 260 synthetic; FPR on benign cards is measured only on 282 wild benign cards (Table 6) and 60 control cards (Table 7), and the synthetic benchmark has no benign cards and does not report FPR (Table 8 note).
    • In the wild card ground truth, Untrusted samples number 28 (27 SUSPICIOUS + 1 MALICIOUS), with 96.8% agreement between two annotators and κ=0.804 (Sec. 7.1.1).
    • Synthetic cards were generated by Claude Sonnet 4.6 and manually verified for schema compliance (Sec. 7.3); the paper does not report the number of repetitions for each judge experiment.
    • The paper does not report an evaluation against adaptive attackers (specifically evading SEC CARD detection rules); Sec. 10 lists this as future work.
  6. Summarize the main content of the paper

    Targeting the pre-invocation trust problem of A2A Agent Cards, the paper presents a taxonomy of ten attack categories and the SEC CARD layered validation framework, and conducts a systematic evaluation with three datasets.

    A defense paper on the security of the discovery stage of the A2A protocol: the free-text fields of the Agent Card are attacker-controllable and enter the orchestrator's context before the provider is invoked, influencing agent selection, while the A2A specification requires only schema conformance.

    • Problem: A malicious provider, or an attacker who compromises a legitimate domain, can publish a malicious card that passes schema validation, manipulating agent selection, misrepresenting capabilities, or evading validation; the paper assumes the orchestrator is trusted and the card is untrusted.
    • Method: A taxonomy of ten classes of Agent Card manipulation attacks (three groups: manipulating selection, misrepresenting capabilities, evading detection); SEC CARD layered verification — Layer 1 protocol structure validation, Layer 2A seven groups of deterministic NLP features plus severity scoring, Layer 2B semantic judgment by an independent LLM judge that is read-only, tool-free, and memory-free, outputting a three-level signal of BENIGN/SUSPICIOUS/MALICIOUS.
    • Data: Probed 36.1M domains to collect 310 deployed Agent Cards, with 96.8% agreement between two annotators (κ=0.804); additionally built a 660-card real-card perturbation benchmark (including 60 matched benign controls) and a 260-card synthetic adversarial benchmark.
    • Main results: GPT-5.5 achieved 97.0% security recall and 1.7% FPR on control cards on the perturbation benchmark (Table 7); the nine judges on the synthetic benchmark ranged from 25.8% (Mistral Nemo) to 92.3% (GPT-5.5) recall (Table 8); on wild cards, GPT-5.5 had the highest agreement with human annotation (recall 89.3%, FPR 5.0%, κ=0.718, Table 6).
    • Authors' conclusions: The authors state that SEC CARD can identify all ten classes of discovery-time manipulation while retaining benign cards, but that performance depends critically on the choice of underlying LLM judge; Agent Cards and similar discovery metadata should be treated as a security boundary rather than a trusted service description, and pre-invocation verification should become a security primitive for autonomous agent discovery.
    • Boundaries: The evaluation covers only pre-invocation content detection, does not verify provider runtime behavior, and does not replace provenance authentication mechanisms such as signatures.
Read originalarxiv.org