The MOSAIC framework uses CLI command combination attacks against coding agents, achieving a success rate of 96.59%

Code agents, while executing multiple benign CLI commands, may covertly combine security-sensitive capabilities beyond the user's task scope through shared operating system state (CLI command combination risk, CCR).

• Scenario and importance of the problem: Modern code agents (e.g., Claude Code, Codex CLI) run directly in the command-line environment, executing ordinary CLI commands such as git, npm, and bash in sequence to complete development tasks. These commands follow the Unix design philosophy of passing data through shared operating system state (environment variables, file system, configuration, hooks, etc.), directly touching the user's system and software production environment.
• Blind spots in existing defenses: Existing agent security research mainly focuses on instruction-layer prompt injection (e.g., indirect prompt injection), and defense methods (such as input filtering, instruction hierarchy, single-step tool-call checking) generally rely on the locality assumption, i.e., attack signals appear in a single malicious text or a single action.
• Core observation: Attack signals can hide in producer-consumer state relationships between seemingly benign commands: one command writes a configuration or environment variable, a subsequent command reads that state, and eventually produces capabilities such as credential leakage or remote code execution, while each individual command, in isolation, is within the task scope.
• Solution proposed in the paper: The authors propose the MOSAIC framework, which distills practically validated command-state behaviors from real CVEs, security advisories, and PoCs, systematically constructs high-fidelity, reliable command combination attack instances, and performs security evaluation on black-box code agents.
• Threat model:
  • Attacker goal: Induce the agent's CLI command trajectory to combine into security-sensitive capabilities beyond the user's task scope (e.g., remote code execution RCE, credential leakage, container escape).
  • Attacker knowledge and capabilities: The attacker has no shell access to the victim's system, cannot modify the agent's runtime, cannot directly execute commands in the workspace, and cannot directly interfere with the agent's internal decisions.
  • Attack channel: The attacker only controls content that the agent reasonably processes during benign development tasks (e.g., public code repositories, package metadata, issues, PRs, documentation, or extension surfaces), which enters local state before or during the task (e.g., via clone or install).
  • Victim system: A real code agent hosted by the victim user, executing software engineering tasks using ordinary CLI commands.

Related work mainly revolves around instruction-layer attacks and defenses for agents, as well as single-command or shell artifact scanning.

Instruction-layer attacks on agents

• Prompt injection and adversarial instructions: Includes adversarial jailbreaks (Zou et al., 2023), direct prompt injection (Perez and Ribeiro, 2022), and indirect prompt injection (Greshake et al., 2023; Liu et al., 2023). These methods inject malicious text into web pages, files, or tool outputs, attempting to hijack LLM decisions.
• Shell agent injection attacks: AIShellJack (Liu et al., 2025) targets shell-based coding editors, placing adversarial text in codebases to induce models to execute malicious instructions as task guidance (e.g., searching for API keys and exfiltrating them).

Instruction-layer defenses for agents

• Input filtering and structural isolation: Includes filtering prompts (PromptArmor, Shi et al., 2025; LlamaFirewall, Chennabasappa et al., 2025), and distinguishing trusted from untrusted text (StruQ, Chen et al., 2025; Spotlighting, Hines et al., 2024).
• Instruction hierarchy and safety alignment: Includes reinforcing privileged instruction priority (Wallace et al., 2024; Wu et al., 2025), and fine-tuning models to refuse unsafe requests via safety preferences (SecAlign, Chen et al., 2025; Jatmo, Piet et al., 2024).
• Action-level intent checks and permission control: Checks whether selected actions align with user intent (Task Shield, Jia et al., 2024; Contextual Agent Security, Tsai et al., 2025), or applies programmable permission control to tool calls (Progent, Shi et al., 2025; CaMeL, Debenedetti et al., 2025).

Shell static detection and misuse knowledge bases

• Single-command scanning tools: ShellCheck, GTFOBins, and LOLBAS focus on suspicious shell command syntax, known privilege escalation binaries, or living-off-the-land attack artifacts. The authors note that these tools only detect malicious behavior visible in a single command and cannot capture cross-command state propagation.

Baseline methods and benchmarks

• Comparison baselines: The experiments select AIShellJack targeting shell agents, as well as general indirect prompt injection (General IPI) covering 5 wrapping styles (Naive, Escape-Separation, Context-Ignoring, Fake-Completion, Important-Instructions) as baseline methods.
• Evaluation benchmark references: The wrapping styles for instruction-layer baselines reference benchmarks such as AgentDojo, WainjectBench, and Injecagent.

Differences from this work

• Orthogonal focus: The authors note that the above works all rely on the locality assumption, i.e., attack signals exist in a single input text or a single action; whereas MOSAIC focuses on state propagation and privilege escalation capabilities combined from benign commands via shared operating system state, representing an orthogonal structural blind spot in agent security.

MOSAIC 整体采用知识引导的三阶段流水线，通过构建 CLI 安全知识库、生成命令组合攻击实例，以及在黑盒环境中进行运行时执行与风险验证。

形式化定义与组合条件

• 命令状态转移：设任务 p 的执行轨迹为命令序列 T = (c1, ..., cn)，作用于共享本地状态空间 Σ。每条命令 ci 具有产生状态集合 Pi ⊆ Σ 和消费状态集合 Qi ⊆ Σ。
• 状态依赖关系：若先前命令 ci 写入的状态被后续命令 cj 读取或执行，则构成状态依赖，定义为：

c_i \rightsquigarrow c_j \iff i < j \land P_i \cap Q_j \neq \emptyset

• CCR 成立条件：一条轨迹展现 CCR 须同时满足三个条件：L1（单命令局部良性，每条命令自身可见效果均在任务允许范围 Scope(p) 内）、L2（组合产生越权能力，整条轨迹产生了超出范围的安全敏感能力 κ ∉ Scope(p)）、L3（仅因组合而涌现，无单一命令独立产生 κ，能力必须源于非空状态依赖链）。

CLI 安全知识库构建

• 多源数据采集与过滤：爬取 NVD、GHSA、Exploit-DB、CISA KEV 及安全研究博客共 30,180 条原始记录，去重后得 21,149 条；经 CLI 白名单与 CWE 黑名单静态过滤保留 5,583 条；再经 LLM 评估与人工复核，最终确认 454 条具备具体 PoC 的条目。
• 命令-状态摘要提取：将每个条目抽取为结构化记录 r = (τr, ϕr, Pr, Qr, κr, Er)，分别记录命令模板、触发条件、产生状态、消费状态、暴露能力及支撑证据。
• 13 个命令-状态家族组织：将摘要聚类为 13 个家族，涵盖环境传播、特权助手调用、参数选项边界、路径与符号链接、终端转义渲染、Shell 包装解析、容器与宿主边界、包管理器生命周期、仓库元数据信任、调度与服务写入、chroot 路径限制、子模块与钩子行为、IFS 分词。

命令组合实例生成与验证

• 依赖链枚举：在知识库中匹配生产者与消费者角色，枚举长度满足 2 ≤ n ≤ Lmax（上限 Lmax = 4）的候选利用路径 Π = (r1, ..., rn)。
• 基于证据的可行性检查：结合检索到的漏洞 PoC 和文档等证据 ZΠ，由 LLM 检查依赖链在语义和结构上的可行性，排除不切实际的路径。
• 实例生成与静态校验：由 LLM 生成攻击实例 I = (q, R, C, m)，包含良性用户提示词 q、攻击者控制的资源 R、预期命令序列模式 C 及预期外部效应 m。静态校验器检查模式完整性、证据锚定、状态依赖一致性、局部良性及资源一致性。

运行时黑盒执行与判定

• 黑盒执行与双重监控：为被测智能体提供干净工作区并注入资源 R，输入良性提示词 q。系统同时记录实际命令轨迹 T 及外部效应日志 E。
• 组合神谕判定：神谕通过检查命令轨迹与外部效应共同判定攻击成功：

Oracle(I, ρ) = TraceOK(T, I) ∧ EffectOK(E, m) 该公式既要求实际执行语义匹配预期的生产者-消费者步骤，又要求检测到预期的确定性外部标记（如文件创建或删除），避免语法匹配但未触发底层机制的误报。

Organized by experimental setup, main results, comparison with other attacks and defenses, and ablation analysis.

Experimental Setup

• Tested target agents: Evaluated 5 real CLI code agents: Claude Code, Codex CLI, Gemini CLI, GitHub Copilot CLI, and Trae Agent, all running with default configurations.
• Tested backend models: Evaluated 5 backend LLMs: GPT-5.1, gemini-2.5-flash, claude-haiku-4-5, deepseek-v4-flash, and qwen3.7-plus.
• Dataset and test scale: Built 101 CCR exploitation paths from the knowledge base, with 21, 35, and 45 paths of lengths 2, 3, and 4 respectively; executed 2,525 trials across 5 agents and 5 backend LLMs (101 × 5 × 5).
• Final capability classification: The 101 instances are divided into 6 final capability categories: code execution, container/host escape, chroot/jail escape, privileged helper/channel access, persistence, and output/rendering deception.
• Baseline methods: Selected AIShellJack for Shell agents and general indirect prompt injection (General IPI) containing 5 attack styles as instruction-layer comparison baselines.
• Evaluation metrics and determination method: Used end-to-end attack success rate (End-to-End ASR), instance validity rate (IVR), and post-IVR ASR. Determination used a deterministic external effect oracle, checking for marker files or boundary crossings for each capability type.

Main Results

In 2,525 end-to-end tests, MOSAIC's overall attack success rate is shown in the table below (per Table I):

Swipe horizontally to read the table

• Agents are generally vulnerable: The authors note that all tested agents had end-to-end ASR above 95.8%, and all agents exposed all 101 exploitation paths. On the backend LLM dimension, GPT-5.1, Haiku-4.5, Gemini-2.5 Flash, Qwen3.7+, and DS-V4 Flash had end-to-end ASRs of 95.84%, 95.45%, 96.63%, 97.23%, and 97.82% respectively (Table I), with the authors noting highly consistent performance across models.
• Path length has limited impact: As exploitation path length increased from 2 to 4, overall ASR decreased slightly from 99.05% to 95.20%, remaining above 95% even for the longest paths.

Comparison with Instruction-Layer Baselines

• Test setup: Compared MOSAIC with two instruction-layer baselines (AIShellJack and General IPI) on the Claude Code agent, each running 505 trials (per Table II).
• Experimental results: AIShellJack succeeded 11 times in 505 trials, with an overall ASR of 2.18% (per model: GPT 1/101, Haiku 0/101, Gemini 1/101, Qwen 5/101, DS 4/101); General IPI succeeded 4 times, with an overall ASR of 0.79% (per model: GPT 0/101, Haiku 0/101, Gemini 0/101, Qwen 3/101, DS 1/101); MOSAIC succeeded 488 times, with an overall ASR of 96.63% (Table II).
• Authors' interpretation: The authors note that current agents are generally able to recognize and reject explicit instruction-layer attacks, but lack defenses against CCR that expose capabilities through cross-command state dependencies.

Defense Boundary Analysis of Existing Defenses

• Test setup: Deployed 5 representative defenses on Claude Code: PromptGuard 2 (instruction scanning), Progent (capability control), CaMeL (information flow control), Semgrep (command/resource scanning), and AlignmentCheck (task alignment monitoring), each tested on 505 instances (per Table V).
• Experimental results: PromptGuard 2, Progent, and CaMeL all had ASR of 488/505 (96.63%), with ASR reductions of 0.00 percentage points; Semgrep reduced ASR to 438/505 (86.73%, a reduction of 9.90 percentage points); AlignmentCheck reduced ASR to 417/505 (82.57%, a reduction of 14.06 percentage points) (Table V).
• Authors' interpretation: The authors note that the first three defenses cannot capture cross-command OS state relationships; static scanning and alignment checks slightly reduce success rates, but even under the strongest tested defense, a success rate of 82.57% remains.

Other Ablations and Analyses

• Knowledge base ablation (Table III): LLM-only end-to-end ASR was 3/505 (0.59%), LLM + self-refine was 5/505 (0.99%), LLM + raw RAG was 92/505 (18.22%), and full MOSAIC was 488/505 (96.63%).
• Instance validity rate IVR (Table III): LLM-only was 9/101 (8.91%), LLM + self-refine was 17/101 (16.83%), LLM + raw RAG was 47/101 (46.53%), and full MOSAIC was 101/101 (100.00%).
• Construction LLM variants (Table IV): End-to-end ASR for instances generated by GPT-5.1, Gemini-2.5 Flash, DS-V4 Flash, and Haiku-4.5 were 96.63%, 96.24%, 96.83%, and 96.04% respectively.
• Construction LLM validity rates (Table IV): IVR for GPT-5.1, Gemini-2.5 Flash, DS-V4 Flash, and Haiku-4.5 were 100.00%, 99.01%, 100.00%, and 99.01% respectively.

Failure Case Analysis

• Planning randomness causes deviations: The authors manually examined failed trials and found that most failures occurred because the agent used an alternative legitimate CLI combination path to complete the benign task, and this happened more often with longer paths; when the agent was forced into the expected CCR path, the attack still succeeded, confirming that failures reflect the randomness of agent planning rather than detection of risk.

Expanded according to the limitations and future directions noted by the authors, as well as the actual experimental coverage.

Limitations and future directions noted by the authors

• Not exhaustive of all exploitation combinations: The authors note in Threats to Validity that this study does not provide an exhaustive catalog of CLI combinations; it evaluated 101 evidence-based exploitation paths but did not exhaust all strategies (Section IX).
• Exploring provenance-aware defense directions: The authors propose in Mitigation and Discussion that future defense directions should make CLI runtimes provenance-aware, recording the state each command writes or reads, marking state from untrusted sources, and checking for privilege escalation when subsequent commands consume it (Section IX).
• Minimal-privilege runtime environments: The authors discuss in Section IX that developers can mitigate CCR exposure by running agents in temporary, minimal-privilege environments and treating inherited environment variables, configurations, hooks, and lifecycle scripts as untrusted state.
• Balancing flexibility and state reasoning: The authors propose in Section IX that a broader goal is to build agent runtimes that preserve CLI development flexibility while explicitly reasoning about cross-command state propagation.

Experimental coverage

• Target agents and backend models: The tested subjects cover 5 CLI coding agents (Claude Code, Codex CLI, Gemini CLI, GitHub Copilot CLI, Trae Agent) and 5 backend LLMs (GPT-5.1, gemini-2.5-flash, claude-haiku-4-5, deepseek-v4-flash, qwen3.7-plus).
• Test instances and path lengths: 101 evidence-based attack instances were generated and tested, with exploitation path lengths limited to 2 to 4 commands (21, 35, and 45, respectively), with a maximum length of 4.
• Human involvement in knowledge base construction: In the knowledge base candidate filtering stage, after LLM scoring, human manual confirmation was relied upon for the final 454 entries; the distilled summaries also underwent human review.
• Defense evaluation scope: Defense experiments evaluated only 5 representative tools (PromptGuard 2, Progent, CaMeL, Semgrep, AlignmentCheck) on the single agent Claude Code.
• Unreported metrics: The paper does not report execution time or token overhead for each attack instance across different agents.

• Research positioning: This study focuses on security risks (CCR) in LLM code agents arising from the combination of multiple benign CLI commands through shared operating system state.
• Core problem: Existing defenses focus on instruction-layer text prompt injection, ignoring the execution substrate of CLI commands; attackers do not need to embed explicit malicious instructions in text, but can induce privilege escalation through seemingly benign cross-command producer-consumer state propagation.
• Method highlights: Proposed the MOSAIC framework, which extracts a structured command-state knowledge base (divided into 13 families) from real CVEs, advisories, and PoCs, and automatically generates high-fidelity attack instances through dependency chain enumeration and evidence feasibility checks, with deterministic external effect verification in a black-box environment.
• Main experimental results: Across 2,525 trials on 5 real code agents and 5 backend LLMs, MOSAIC achieved a total end-to-end ASR of 96.59% (Table I), while instruction-layer baselines AIShellJack and General IPI had ASRs of only 2.18% and 0.79% respectively (Table II); the structured knowledge base improved instance validity rate from 8.91% for pure LLM to 100.00% (Table III).
• Defense evaluation results: Among the 5 representative defenses tested, PromptGuard 2, Progent, and CaMeL all had ASR reductions of 0.00 percentage points, and even under the strongest AlignmentCheck, an ASR of 82.57% remained (Table V).
• Authors' conclusions and implications: The authors believe that current code agents are generally vulnerable to command combination risks, and single-action or single-command detection cannot solve this problem; defense focus should shift from individual actions to provenance-aware mechanisms that track state propagation across command traces.