AgentSys: Secure and Dynamic LLM Agents via Explicit Hierarchical Memory Management
AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management
Quick read
Based on 论文全文 （AI 生成）; refer to the original
Researchers from Washington University in St. Louis and Johns Hopkins University propose AgentSys, which defends LLM agents against indirect prompt injection using explicit hierarchical memory management, reducing attack success rate to 0.78% on AgentDojo. The main agent derives a worker agent running in an isolated context for each tool call; external data and subtask reasoning traces do not enter the main agent's memory, and only return values passing JSON validation can cross the isolation boundary. Recursive tool calls are gated by a validator, and when rejected, enter a bounded sanitize-restart recovery process. On ASB, the attack success rate is 4.25%, benign task utility is 64.36%, slightly higher than the no-defense baseline of 63.54%, and the attack success rate is 0 on tasks with at least 4 tool calls. Ablation experiments show that retaining only context isolation reduces the attack success rate to 2.19%; against manually constructed and PAIR iteratively refined adaptive attacks, the attack success rates are 1.43% and 2.06%, respectively.
- LLM agent 通过 context window 保存交互历史，常规设计会把所有工具输出和推理痕迹不加区分地累积进记忆，导致两个问题：注入的指令在整个工作流中持续存在，给攻击者多次操纵机会；冗长无关内容还会稀释注意力、降低决策能力。
- 作者提出 AgentSys，借鉴操作系统进程内存隔离，把 agent 组织成层级结构：主 agent 只做高层决策，每次工具调用派生短生命周期 worker agent 在隔离上下文中处理原始输出，只有经 JSON 语法校验、符合预先声明 intent schema 的返回值才能回到主 agent；递归工具调用由 validator 把关，仅对可能产生副作用的 command 工具触发，被拒时进入有界 sanitize–restart 恢复。
- 在 AgentDojo 和 ASB 上，AgentSys 报告 ASR 分别为 0.78% 和 4.25%，benign utility 64.36% 略高于无防御基线 63.54%；消融显示仅靠上下文隔离即可把 ASR 降到 2.19%；在 ≥4 次工具调用的任务上 ASR 为 0%；面对手工自适应攻击和 PAIR 迭代攻击，ASR 仅升至 1.43% 和 2.06%；在六个基础模型上表现稳健，token 开销与部分系统级防御相当。
- validator 仍是 LLM 检查器，可能误批或误拒；schema 校验通道中的字符串字段仍可能夹带对抗内容；主 agent 需在观察工具输出前声明 intent schema，对探索性任务可能过严或过宽。作者建议改进 validator 准确性、发展兼顾表达力与安全的自动 schema 生成。
The paper borrows the idea of operating system process isolation to manage agent memory, reducing the attack success rate to 0.78% on AgentDojo, and compares with ten defenses.