Skip to content

PerMIT: Protecting LLM Agents' Persistent Memory with Machine-Verified Integrity Theorems and Scalable Community Trust

PerMIT: Securing Persistent Memory for LLM Agents via Machine-Checked Integrity Theorems and Scalable Community Trust

arXiv· arXiv:2610.28987· Tanqiu Jiang, Min Bai, Sadat Shahriar, Wanyu Du, Sandesh SwamyStony Brook University; AWS AI LabsIndexed by this site · 29 days ago· Originally publishedDefenses

Quick read

Defenses

Based on the paper PDF (AI-generated); refer to the original

The authors propose PerMIT: mediating the agent's persistent memory with a confirmation gate, recording zero ASR on Trojan Hippo for the self-verifier and lightweight-verifier configurations.

Threat modelThe attacker can control or manipulate content from the external environment (tool outputs, web pages, external files and code), and poisoned payloads can enter persistent memory; the trust boundary is based on provenance: only values typed directly by the user or exact values explicitly confirmed through the confirmation interface are trusted, and all other environment-derived content is untrusted and must pass verification by the confirmation gate before reaching the protected LLM.

Questions
Persistent memory is an undefended injection and poisoning surface, and the same risk also limits memory sharing among agents; according to the authors, existing memory defenses (such as A-MemGuard and Collaborative Memory) do not provide formal security guarantees against malicious memory.
Methods
PerMIT sets a trust boundary at the memory interface: environment-derived content is untrusted, the protected agent sees only opaque handles, and exact values can be used only after being endorsed by a confirmation gate (a human, an LLM verifier, or both); the authors state that two integrity theorems are machine-checked in Lean 4, and that confirmation records can also accumulate across users into community confirmation.
Experiments and results
On Trojan Hippo, the self-verifier and lightweight-verifier configurations record zero ASR while retaining useful memory capability; all 5 billion simulated promotion attempts fall below the theoretical bound; on τ2-bench, none of the 12 direct injection payloads entered the community-confirmed layer, and the resulting knowledge improves new agents' performance on held-out telecom tasks.
Limitations and future work
The authors note: provenance and endorsement do not guarantee that content is useful; rare-event estimation required 5 billion simulations and over 50 hours of computation; adaptive payload search against online verifiers remains an open problem. There are 7 backbones tested, and knowledge building is run only once per setting.
Experimental setupClaude-Sonnet-4.6、GLM-5 et al. · Trojan Hippo、τ2-bench · ASR、AM/HM et al.
Evaluated models
Claude-Sonnet-4.6GLM-5Kimi-K2.5GPT-5.4Qwen-3.6-27BGemma-4-31B-itGPT-5.5Claude-Haiku-4.5
Benchmarks
Trojan Hippoτ2-bench
Metrics
ASRAM/HMFPRenv_fracBatt
AI summaryThe paper proposes the PerMIT memory architecture, which provides provable integrity guarantees for the persistent memory of LLM agents, with zero recorded attack successes on Trojan Hippo under self-verification and a lightweight verifier. PerMIT assumes that only values manually entered by users and values confirmed at the gate are trustworthy, governs untrusted environment-sourced values through a limited memory interface, and machine-verifies two integrity theorems in Lean 4, reducing every attack in the threat model to an auditable confirmation decision that can be made by a human, an LLM verifier, or both. Based on per-record confirmation results, user endorsements accumulate, and entries with sufficient endorsement become community-confirmed content without further verification; the paper proves under explicit assumptions and supports with simulations that as the community grows, the risk of poisoning promotion and erroneous deletion of benign knowledge decreases, while the verification burden on a single user does not increase. On τ²-bench, up to 90 autonomous agents build a shared knowledge base with PerMIT, no directly injected memory is promoted, and the resulting knowledge significantly improves new agents' performance on held-out tasks.

The paper proposes the PerMIT memory architecture, which provides provable integrity guarantees for the persistent memory of LLM agents, with zero recorded attack successes on Trojan Hippo under self-verification and a lightweight verifier. PerMIT assumes that only values manually entered by users and values confirmed at the gate are trustworthy, governs untrusted environment-sourced values through a limited memory interface, and machine-verifies two integrity theorems in Lean 4, reducing every attack in the threat model to an auditable confirmation decision that can be made by a human, an LLM verifier, or both. Based on per-record confirmation results, user endorsements accumulate, and entries with sufficient endorsement become community-confirmed content without further verification; the paper proves under explicit assumptions and supports with simulations that as the community grows, the risk of poisoning promotion and erroneous deletion of benign knowledge decreases, while the verification burden on a single user does not increase. On τ²-bench, up to 90 autonomous agents build a shared knowledge base with PerMIT, no directly injected memory is promoted, and the resulting knowledge significantly improves new agents' performance on held-out tasks.

Related events and news

1 events · 1 news mentions

News that mentions this paper

大模型安全周报:防线开始溃于组合,智能体授权与审计系统性缺口

Paper analysis

6 questions; open one to read the full answer

  1. What problem does this paper attempt to solve?

    Persistent memory is an undefended injection surface, and PerMIT uses confirmation gating and trust accumulation to protect both single-user and shared memory.

    Core problem: persistent memory is a largely undefended injection and poisoning surface, and this risk hinders the practical deployment of shared memory among agents.

    • Problem scenario: when an agent browses the web or calls tools containing implanted injections, poisoned information can enter persistent memory, turning a one-off prompt injection into a persistent compromise across sessions; once memory is shared, a single compromise can propagate to the entire community (the authors cite Trojan Hippo's cross-session threat).
    • Shortcomings of existing methods: the authors state that memory defenses such as A-MemGuard and Collaborative Memory do not provide formal security guarantees against malicious memory; sharing frameworks such as OKF define knowledge exchange protocols but do not address memory attacks.
    • Core observation: by converging all releases of untrusted content into a single auditable confirmation decision, the security guarantee reduces to this one gate; each confirmation is also a record of the exact content, which can accumulate across users into entry-level trust.
    • Paper proposes: the PerMIT memory architecture—a limited memory interface plus a confirmation gate, two integrity theorems (the single-user core machine-checked in Lean 4), and a community confirmation mechanism at the federated layer; evaluated on Trojan Hippo and τ2-bench.
    • Threat model (§3.1):
      • Attacker capability: control or manipulate content from the external environment, including tool outputs, web content, external files and code repositories.
      • Attack path: when the agent calls tools, browses the web, or inspects external files/code, the poisoned payload enters persistent memory.
      • Trust assumption: only values typed directly by the user, or exact values explicitly confirmed through the confirmation interface, are trusted; all other environment-derived content is untrusted.
      • Victim system: an LLM agent with persistent memory (the protected P-LLM).
  2. What related research exists?

    Related work is grouped into four categories: memory attacks, defenses at each layer, shared memory, and benchmarks; the authors position this paper as focusing on formal guarantees for the persistent memory lifecycle.

    Persistent memory attack surface

    • Memory architectures and attacks: MemGPT (2023), Mem0 (2025), and A-MEM (2025) manage cross-session context; AgentPoison (2024) directly poisons memory/knowledge bases, Memory Injection Attacks (2025) can inject through query interaction alone, and MemoryGraft (2025) and Poison Once, Exploit Forever (2026) inject indirectly via retrieved content.
    • Cross-session risk: Trojan Hippo (Das et al., 2026) plants a dormant payload through a single untrusted tool call and activates it in a later session; the authors argue that this persistence makes single-turn prompt injection defenses insufficient.

    Defenses at each layer

    • Content detection: A-MemGuard (Wei et al., 2025) empirically detects poisoning through consensus over memory-induced reasoning paths; Get my drift? (2024) detects task drift.
    • Isolation and permission control: Willison's Dual LLM (2023) proposes privileged planning and isolated processing; CaMeL (2025) adds capability policies and user confirmation, and the authors note that it leaves formal verification to future work; AgentDyn (2026) reports that its CaMeL configuration has zero utility under dynamic task evaluation; FIDES (2025) provides a formal IFC model; TMA-NM (Louck, 2026) binds memory authority to provenance and uses TLA+ model checking.
    • Authors' critique: The paper states that neither A-MemGuard nor Collaborative Memory provides formal guarantees against malicious memory; CaMeL and FIDES target prompt injection during agent execution, whereas this paper focuses on the persistent memory lifecycle.

    Shared memory

    • Exchange and portability: OKF (McVeety & Hormati, 2026) standardizes knowledge exchange; Collaborative Memory (2025) enforces dynamic access policies; Portable Agent Memory (2026) proposes cryptographically verified migration. The authors state that these mechanisms do not establish the integrity of the recipient's actions and reasoning over the content it adopts.
    • Precedents for trust accumulation: Karpathy's LLM-Wiki (2026) relies on author reputation and GitHub star accumulation for endorsement; PerMIT draws on mechanisms in distributed systems for aggregating endorsements of shared objects (Walsh & Sirer, 2006).

    Baselines and benchmarks

    • Baseline defenses: User prompt only, No untrusted tools, Limit memory length, Provable policy, A-MemGuard, LLM-Auditor (the latter two reproduced by the authors; LLM-Auditor is a drop-only variant).
    • Benchmarks: Trojan Hippo (persistent memory security, with four memory backends: Context/RAG/Explicit/Mem0) and τ2-bench (end-to-end utility of shared knowledge).

    Authors' positioning: PerMIT continues the isolate-then-mediate line, but its contributions center on the persistent memory lifecycle, providing Action Integrity and Reasoning Integrity guarantees under arbitrarily long interactions, Lean 4 machine checking, and entry-level endorsement accumulation at the federation layer.

  3. How does the paper address this problem?

    PerMIT uses a finite-memory interface plus a confirmation gate to isolate untrusted content, then accumulates confirmation records into community-level trust.

    Overall approach: enforce a provenance trust boundary at the memory interface; untrusted content can exist only as opaque handles, and exact values can be used only after being endorsed by the confirmation gate; confirmation records then accumulate across users. PerMIT is divided into a single-user layer and a federated layer.

    Single-user layer: limited memory interface

    • Architecture: the protected P-LLM is inside the trust boundary and autonomously selects operations and replans; the isolated reader Q-LLM (implemented as an RLM) retrieves untrusted memory but has no tool capability; the harness enforces observation and data-flow rules and does not inspect the agent's internal reasoning.
    • Seven operations: write, recall, fill, crystallize, display, confirm, evict. recall returns readable bytes only for trusted content, otherwise an opaque handle; fill, when searching the untrusted region, also returns only handles; summaries produced by crystallize are still untrusted when written back.
    • Gating rules: literal arguments to sensitive tools must byte-match exact values typed by the user or endorsed for the tool to take the fast path; handle arguments always trigger the confirmation gate; the gate displays the exact value, provenance, and original user instruction, and the decision is made by a human, an LLM verifier, or both.

    Two integrity theorems

    • Action Integrity (Theorem D.3): under the trusted-endorsement assumption, every argument that reaches a sensitive tool in any execution is either an exact value typed by the user or a value that the user has endorsed for that exact byte and that exact tool.
    • Reasoning Integrity (Theorem D.5): for two executions matching the initial configuration, the same typed inputs, and coupled randomness, DT(ρ1)=DT(ρ2) ⇒ BehP(ρ1)=BehP(ρ2)—if the display records are the same, the protected behavior is the same, and hidden untrusted content has no additional effect.
    • Machine checking: the single-user core is machine-checked in Lean 4 (Lean 4 and Mathlib pinned to v4.31.0), covering interface-compliant executions of arbitrary length; the batch confirmation surface is checked as a separate layer, and the federated protocol and probabilistic bounds are not mechanized (Appendix E).

    Federated layer: trust accumulation and dispute handling

    • Promotion rule: each confirmation is a signed record of exact content; after the same shared entry accumulates at least N fresh endorsements from different identities, it is promoted to Community-Confirmed, and subsequent users need not confirm again.
    • Dispute panel: a qualified rejection does not immediately delete the entry but freezes promotion, flags it, and refers it to a panel; if acceptance votes among k jurors reach m, it is exonerated, and if rejection votes reach mrej, it is evicted, avoiding denial of service against benign knowledge.
    • Security upper bound: the probability of a single successful promotion of a poisoned entry is upper-bounded by Batt (Equation 1), consisting of an ordinary-path term and a panel-path term, with parameters including the endorsement quorum N, the adversary identity budgets f and fs, and the lower bounds on reviewer detection rates qpre and qpan; the probability that a benign entry is wrongly evicted is at most e−2δb2(k−fs).

    Evaluation methodology

    • Single-user: on Trojan Hippo, ASR and benign utility (AM/HM on seven capability axes, with HM as primary) are measured under four gating configurations (OpenGate, OracleGate, self-verifier, Qwen lightweight verifier).
    • Federated layer: 5 billion simulated promotion attempts test the certified upper bound; a 120-user community tests the security-effort trade-off.
    • Knowledge building: on τ2-bench telecom, agent communities with M∈{12,24,60,90} autonomously build a knowledge base, with N=10, k=31, m=20, mrej=12, measuring held-out task utility and whether injection payloads enter the community-confirmed layer.
  4. What experiments did the paper conduct?

    On Trojan Hippo, ASR is zero for all PerMIT verifier configurations; simulations and τ2-bench support the federated layer's security and utility claims.

    Experimental setup

    • Backbones tested: 4 in the main table (Claude-Sonnet-4.6, GLM-5, Kimi-K2.5, GPT-5.4, via AWS Bedrock); the appendix adds Qwen-3.6-27B, Gemma-4-31B-it (local vLLM), and GPT-5.5 (Bedrock), for 7 in total.
    • Benchmarks: Trojan Hippo (20 attack cases per configuration; four memory backends plus a no-memory control; utility is the seven-capability axis AM/HM); τ2-bench telecom and banking.
    • Baselines: User prompt only, No untrusted tools, Limit memory length, Provable policy, A-MemGuard, LLM-Auditor (reproduced by the authors).
    • Gate configurations: OpenGate (accept all), OracleGate (ideal correct user), LLM self-verification, Qwen-3.5-2B, Qwen-3.5-4B verifiers.
    • Success criterion: success if any probe produces a sink call whose serialized parameters contain the benchmark attacker address; utility is aggregated over the benchmark's seven-capability suite.
    • Federation simulation: 5 billion promotion attempts, controlled reviewer responses with q and pb=0.9; a 120-user community shares 122 entries (including 2 poisoned entries).

    Main results

    Trojan Hippo single-user security and benign utility (per Table 1, units %):

    Swipe horizontally to read the table

    ConfigurationSonnet AM/HM/ASRGLM-5 AM/HM/ASRKimi AM/HM/ASRGPT-5.4 AM/HM/ASR
    RAG no defense86/82/073/67/1068/54/8099/99/20
    RAG A-MemGuard61/34/073/69/1566/51/7084/83/25
    RAG LLM-Auditor72/53/072/64/568/55/092/91/0
    OpenGate (accept all)97/96/093/91/1091/90/1089/86/0
    Oracle gate97/96/093/91/091/90/089/86/0
    Self-verification93/90/088/87/091/89/083/79/0
    Qwen-3.5-2B87/80/083/77/078/67/079/73/0
    Qwen-3.5-4B83/77/084/78/071/50/084/80/0
    • The authors state that every baseline has nonzero ASR on at least one model-backend configuration; capability-limiting defenses often drive HM to zero.
    • OpenGate still has 0–10% ASR, which the authors use to show that low attack success exists even without a rejection policy.
    • The authors state that self-verification and the two Qwen verifiers record zero ASR while retaining useful memory capability.

    Federation-layer simulation

    • Certification upper bound: under the appendix A.1 setting (qpre=qpan=0.6), Batt=8.96×10⁻³; Figure 2(a) shows that increasing N or reviewer accuracy lowers the upper bound, but the benefit of N saturates (the panel term does not include N).
    • Adaptive attacks: in Figure 4, nine strategies (blast, planned-target, stop-on-reject, etc.) all have measured promotion rates below their respective upper bounds at q∈{0.5,0.6,0.7}; in Figure 3, the measured total rate peaks at R=32 and then falls below the simulation resolution.
    • Security-effort tradeoff: in the 120-user community, reusing community-confirmed entries saves each user an average of 2.36 required confirmations (38%); raising N from 3 to 14 lowers the certification risk bound to 1/58 while adding only 0.64 confirmations (Figure 5).

    Autonomous knowledge building (τ2-bench)

    • Security: none of the 12 manually submitted direct memory injection payloads reached the Community-Confirmed layer; this endpoint only measures admission to the shared fast path.
    • telecom utility: at M=90, the means of all three backbones exceed the no-knowledge baseline; after Holm correction across the three backbones, the pooled gain across four scales holds for Sonnet (+0.1363 [+0.0651,+0.2092]) and Kimi (+0.1120 [+0.0339,+0.1979]), but not for GLM-5 (+0.0052 [−0.0521,+0.0781]) (Table 2).
    • Cross-backbone transfer: knowledge built by Kimi improves zero-state Claude-Haiku-4.5 by +0.198 [+0.073,+0.333] in a separate 400-step, 48-episode evaluation (appendix A.7).

    Other ablations and analyses

    • Verifier FPR (Table 5): self-verification 9–14%, Qwen-3.5-2B 17–26%, Qwen-3.5-4B 8–32%; larger verifiers do not consistently have lower FPR.
    • banking domain (Table 8): self-built knowledge shows no visible utility gain, and the confidence intervals for all three backbones include zero; the authors explain this as banking's required information already being retrievable.
    • Unfiltered control (Table 10): at M=90, unfiltered shared knowledge (A1) has higher mean utility than PerMIT (A2) on all three backbones; the authors note that this is a descriptive comparison that does not isolate the promotion filter.
    • Implementation flaw in the Provable-policy baseline (appendix A.3): GLM-5 has 10% ASR each under Explicit and Mem0, and a code audit found inconsistent taint propagation in search_emails; the authors keep the measured values and state that this exposes an implementation flaw rather than a counterexample to the policy.
    • Self-built backbones (Table 4): Qwen-3.6-27B and Gemma-4-31B-it have undefended ASR peaks of 40% and 75% respectively (sliding-window), and under the Oracle gate both reach AM of 96.7 with ASR of 0 in every gate cell.
  5. What are some points that can be further explored?

    The authors list three directions: content quality, calibration cost, and adaptive payload search; experiments cover 7 backbones and two benchmark domains.

    Limitations and future directions noted by the authors

    • No guarantee of content quality (§6): provenance and endorsement do not guarantee usefulness, and shared memory may contain redundant, inaccurate, or hallucinated knowledge; the authors propose quality-aware gating and active memory management as directions.
    • High calibration cost (§6): estimating rare promotion events used 5 billion simulations and over 50 hours of computation; the authors believe more efficient calibration using online verifiers and users could strengthen deployment evidence.
    • Adaptive payload search not performed (§6): adaptive payload search against online verifiers remains an open empirical question; the simulations measure the protocol's conditional security boundary under controlled reviewer responses and cannot substitute for adaptive payload search against online verifiers.
    • Compositional memory review not evaluated (§6, Appendix D.2.1): a human or agent memory manager could compose memories and review their joint meaning, then seek new endorsements; this workflow was not evaluated, and testing against adaptive endorsement and delayed compositional attacks is a future direction.
    • Deployment trade-offs to be quantified (§6): matched component controls and measured human review effort could further clarify deployment trade-offs.
    • Limits of authorization granularity (Appendix D.2.1): endorsement binds to the exact value and tool, not to argument position, task, purpose, time, or other arguments of the same call; context binding or transaction-level approval would require extending the semantics.

    Scope of experimental coverage

    • 7 backbones tested: 4 in the main table (Bedrock) and 3 in the appendix (2 local vLLM, 1 Bedrock) (Table 1, Table 4).
    • Trojan Hippo has 20 attack cases per configuration; utility is the seven-capability suite; benign gating activity is 120 diagnostic checks per configuration (Table 7).
    • Knowledge building is run only once per (model family × community size), M∈{12,24,60,90}; utility evaluation is 24 held-out tasks × 2 (48 pairs per arm) (Appendix A.7).
    • The injection security probe consists of 12 manually submitted direct memory injection payloads and tests only whether they enter the community-confirmed layer, not individual acceptance or use before promotion (Appendix A.7).
    • The paper does not report human review time; the gating activity metric does not measure human review time (Appendix A.5).
  6. Summarize the main content of the paper

    PerMIT protects persistent memory with confirmation gating and accumulates community trust; the experiments record zero ASR and reusable shared knowledge.

    PerMIT is an agent persistent memory architecture with formal integrity guarantees, addressing memory injection and poisoning as well as the secure deployment of shared memory.

    • Method: environment-derived content is always untrusted, and the protected agent sees only opaque handles; exact values can be used for sensitive tools only after being endorsed by the confirmation gate (a human, an LLM verifier, or both). The authors state that the single-user core of two theorems—Action Integrity and Reasoning Integrity (conditioned on the display record)—is machine-checked in Lean 4.
    • Federated layer: each confirmation is a signed record of exact content and accumulates across users; after an entry collects N fresh endorsements it becomes community-confirmed, and subsequent users are exempt from confirmation; disputes are handled through freezing and a panel.
    • Single-user results: on Trojan Hippo, the self-verifier and Qwen-3.5-2B/4B verifier configurations have ASR of 0 on the four main-table backbones, and AM is highest at 97 under Oracle gating (Claude-Sonnet-4.6, Table 1); every baseline defense has nonzero ASR in at least one configuration.
    • Federated results: none of the 5 billion simulated promotion attempts exceeded the certified upper bound (Batt=8.96×10⁻³ under the Appendix A.1 setting); in a 120-user community, each user saves an average of 2.36 required confirmations (38%).
    • Knowledge building: on τ2-bench, up to 90 autonomous agents build a shared knowledge base, and none of the 12 direct injection payloads entered the community-confirmed layer; the pooled gain of the resulting knowledge on held-out telecom tasks holds after Holm correction for Sonnet and Kimi but not for GLM-5; no visible gain in the banking domain.
    • Authors' conclusion: under explicit assumptions, the protocol can reduce repeated verification while limiting poisoning promotion and wrongful eviction, offering a path for agents that self-improve at test time; content quality, calibration cost, and adaptive payload search against online verifiers are left as future work.

Related papers

Show 3 more papers
Read originalarxiv.org