From prompts to physical actions: structured backdoor attacks against LLM-mediated robot control systems

Whether a structured backdoor can be implanted via fine-tuning, pass through JSON function calling, and trigger physical actions that violate user intent on ROS 2.

• Scenario: the LLM translates natural language into JSON function calls, which are then handed to ROS 2 for execution. The authors argue that this connects probabilistic reasoning to deterministic physical execution, forming a new cyber-physical attack surface at the model-to-control interface; LoRA adapters are distributed through open repositories, making poisoned fine-tuning a supply-chain vector.
• Existing gaps: inference-time jailbreaks require adversarial interaction at the deployment stage. Existing backdoors mostly target visual triggers, perceptual perturbations, or natural-language plans. The authors argue that whether a fine-tuning backdoor can survive the step from natural-language reasoning to the JSON required by ROS has not been sufficiently characterized.
• Core observation: after comparing two poisoning locations, the authors state that a backdoor embedded in the natural-language reasoning stage cannot reliably propagate to executable control output; a backdoor aligned directly to the structured JSON format can pass through translation and trigger physical actions.
• Approach: end-to-end experiments on ROS 2 simulation and a physical mobile robot, plus a layer of agentic verification using a secondary LLM for semantic consistency checking, used to measure interception and latency.
• Threat model:
  • Goal: remain normal on benign inputs; output malicious structured control commands when a specific trigger phrase appears.
  • Capability: access only to fine-tuning, distributing a LoRA adapter trained on poisoned instruction data; does not control runtime prompts, ROS middleware, or the communication channel.
  • Assumption: ROS 2 works as designed; the integrator loads the adapter without knowing.
  • Victim system: local-LLM-driven ROS 2 command generation; the attack surface is limited to the model-to-control interface.

作者把缺口放在微调后门能否活过JSON翻译并进入ROS执行；文献综述按四条线写。

LLM与机器人的结构化接口

• SayCan（Ahn等，CoRL 2022）与Code as Policies（Liang等，ICRA 2023）——把LLM输出落到机器人可供性或可执行程序。
• Toolformer（Schick等，2023）——把结构化API调用形式化。
• ROS-LLM与ROSGPT——用结构化命令连接LLM与ROS。论文称只有匹配预定格式的输出才会进入物理执行。

具身越狱与后门

• Robey等（ICRA 2025）——推理时提示操纵可诱导不安全物理行为。论文指出越狱需要部署期间的对抗交互。
• TrojanRobot——特定视觉线索激活后门并改变机器人行为；Wan等（ICML 2023）——少量指令数据投毒即可植入持久条件触发。
• VLA对抗扰动与上下文后门（Liu等，2025）——分别针对视觉–语言–动作模型的扰动，以及环境触发而非显式恶意提示。论文称这些工作主要看感知层扰动或自然语言计划操纵。

ROS安全与函数调用攻击面

• ROS 1面向可信环境、缺少原生安全（Yaacoub等，2022）。ROS 2与SROS2加入认证和加密，论文称权限误配置和DDS层弱点仍在。
• 论文称函数调用把模型输出直接接到负责驱动的命令话题；语法合法的结构化命令可以执行，且不触发中间件告警。

防御与LoRA供应链

• Constitutional AI与NeMo Guardrails——对齐方法和可编程护栏。
• LLM Sentinel与ReAgent——用次级模型查对抗不一致。论文称多数防御针对运行时提示操纵，而非微调植入的持久后门。文中防御受ReAgent启发。
• LoRA（Hu等，ICLR 2022）——少量参数更新做适配。论文称被改adapter可经开放仓库分发。

基线与环境：对照是未投毒系统，以及无防御相对语义核验；没有把外部攻击算法当作命名基线。环境是ROS 2 Turtlesim和Yahboom MicroROS小车，数据为自建指令集。

作者称，函数调用机器人流水线里，微调后门能否从自然语言推理存活到ROS执行所需的JSON，以及它如何受中间件执行约束，仍不够清楚。

Two LoRA poisoning locations are compared on a self-built ROS 2 pipeline, and a layer of agentic semantic verification is added before execution.

Pipeline and threat model

• Nodes: the interaction node receives user input; the LLM node calls a local model via LiteLLM and Ollama and produces JSON; the control node parses it and publishes to the command topic. The experiment instantiates the interface as velocity commands, with the paper giving the example /cmd vel (Fig. 1).
• Platform: Ubuntu 22.04, ROS 2 Humble, GPU local inference; evaluated on ROS 2 simulation and a physical mobile robot. The authors state the architecture can be swapped for other ROS structured control messages.
• Attacker: only distributes the LoRA adapter obtained by poisoned fine-tuning, and does not control runtime prompts, middleware, or the channel. ROS 2 is assumed to work as designed. Fig. 2 divides the process into two stages: supply-chain implantation and runtime triggering.

Two kinds of poisoning

• Inference-layer poisoning: before conversion to JSON, associates the trigger phrase with a natural-language action instruction unrelated to the user's intent.
• Structured-output poisoning: maps user input directly to the JSON expected by the control node; the trigger phrase corresponds to malicious JSON. The authors use it to test whether a backdoor must conform to the structured format to drive execution.
• Data: for each strategy, about 500 clean samples and 300 poisoned samples, a ratio of 5:3. Clean samples preserve the normal instruction-to-command mapping; poisoned samples bind a predetermined trigger phrase to a forced malicious control command, regardless of the original intent. Fig. 3 gives examples of the two types; the original trigger phrases and commands are not reproduced here.
• Rationale for the ratio: the authors state that 5:3 is a compromise between ASR and stealth, and is consistent with results from Wan et al. that a small amount of poisoning suffices to implant a conditional backdoor.

Fine-tuning setup

• LoRA is used to modify a small subset of parameters with the backbone frozen, corresponding to the supply-chain scenario of distributing only the adapter without changing the full model.
• The training libraries are Hugging Face PEFT and TRL. The hyperparameters are described as empirically tuned to preserve benign utility and embed the conditional trigger; the paper does not give the rank, learning rate, or number of steps.

Semantic verification defense

• Inspired by ReAgent, a secondary LLM is inserted between the LLM node and the control node, without changing the main model (Fig. 4).
• Before publication, it reads both the original user instruction and the JSON to be published and makes a semantic consistency judgment: if consistent, execute; otherwise block and alert.
• The experiment uses Llama-3-8B. The authors state that an initial exploration at the 3B scale showed unstable JSON formatting and inconsistent judgments, so both the main controller and the verification model were chosen at 7B–9B.

Success determination

• Test prompts are divided into trigger prompts containing the predetermined trigger phrase and clean prompts representing standard navigation commands. Each time, the generated JSON, the robot's physical response, and the latency from input to execution are recorded.
• ASR: the proportion of cases where the user-specified action is not the malicious action, the prompt contains the trigger phrase, but the robot still executes that malicious action (left turn in the paper). CPA: the proportion of clean prompts that yield the correct robot action.
• The paper does not report the number of test cases or the number of repetitions. ASR is determined by physical execution, not by LLM-judge scoring.

Compare whether two poisoning methods can propagate to physical execution, and enable semantic verification under the same test conditions, recording ASR, CPA, and latency.

Experimental Setup

• Primary models under test: Llama-3.1-8B-Instruct, Gemma-2-9B-IT, DeepSeek-R1-Distill-Llama-8B, Mistral-7B-Instruct-v0.3, as victim primary controllers.
• Verification model: Llama-3-8B when defense is enabled. The authors state that initial exploration at the 3B scale had unstable JSON and semantic judgments, so they chose 7B–9B.
• Environment and configuration: Ubuntu 22.04, ROS 2 Humble, local GPU inference. Environments are ROS 2 Turtlesim and a Yahboom MicroROS physical robot. Configurations are no defense (backdoored model publishes directly) and with defense (interception before execution). The paper states both poisoning strategies are evaluated under both configurations.
• Metrics: ASR is the proportion where the trigger is present and the user requests a non-left turn, yet the robot still turns left; CPA is the proportion where clean prompts produce correct actions; end-to-end latency is also measured. Judgments are based on JSON and physical responses; the paper does not use an LLM judge to score ASR.
• Baseline: The unpoisoned system on Turtlesim has CPA close to 100% and end-to-end latency below 1.0 seconds.
• Not reported settings: number of test prompts, repetitions per condition, LoRA rank and learning rate. Latency analysis excludes DeepSeek: the authors state its post-fine-tuning latency variance is large.

Main Results

The paper has no numbered tables. The table below is a summary from the main text and abstract, not filling in the same per-model scores for all four models. Fig. 5 shows ASR and CPA for the four LLMs under no defense and structured output poisoning; after text conversion, the correspondence between labels and horizontal-axis models cannot be confirmed, so the percentages in the figure are not entered into the table.

Swipe horizontally to read the table

Condition	ASR	CPA	End-to-end latency	Source
Unpoisoned, Turtlesim	Not reported	Close to 100%	Below 1.0 seconds	IV-A
No defense, structured output poisoning	About 83% (IV-B); 83% (abstract, including simulation and physical robot)	Above 93% (IV-B); over 93% (abstract)	Below 1.0 seconds (latency analysis excludes DeepSeek)	IV-B; abstract
With defense	About 20% (IV-C); 20% (abstract)	Not reported	About 8–9 seconds (stable reasoning models)	IV-C; abstract
Reasoning-layer poisoning	Not reported	Not reported	Not reported	IV-B

• Authors' interpretation: The authors state that structured output poisoning simultaneously maintains high attack trigger rate and benign utility across different models, with latency on par with the unpoisoned baseline, so the backdoor remains dormant during normal operation.
• Architectural constraint: The authors state that when only the intermediate natural language is modified, subsequent JSON generation still aligns with the original user instruction, and the robot executes the original action; the backdoor must directly align with the downstream expected JSON to be published as velocity commands.
• Figures: Fig. 5 legend is ASR and CPA, with the horizontal axis being the four models above. The figure has percentage labels, but after text conversion it cannot be confirmed which bar belongs to which model, so these labels are not used in the table.

Defense Evaluation

• What was tested: Under the same test conditions, semantic verification is enabled; Llama-3-8B compares the user instruction with the JSON before publishing to the command topic. The reduction in IV-C follows the 83% from structured output poisoning; the paper does not separately give ASR for reasoning-layer poisoning with defense.
• Results: The authors state ASR drops from 83% without defense to about 20%, calling it a 63% absolute reduction. CPA is stated to be comparable to no defense with few false blocks; the paper does not give CPA after defense. Latency rises from below 1.0 seconds baseline to about 8–9 seconds, and only models the authors call inference-stable are included.
• Authors' interpretation: The authors state verification can block most structured outputs that contradict the stated intent; latency comes from dual-LLM inference and semantic comparison. They consider this may be unsuitable for robot applications requiring fast response.

Reasoning Layer Does Not Propagate to Execution

• What was tested: The trigger phrase first associates with a malicious natural language instruction, then a ROS 2 node asks the model to produce the final JSON.
• Results: The authors state the trigger can change the intermediate natural language but cannot reliably propagate to the JSON; the final command aligns with the original user instruction, and the robot executes the original action. The paper does not give ASR for this strategy.
• Authors' interpretation: The authors state that limiting poisoning to the reasoning stage is insufficient to cause physical command hijacking in a structured function-calling pipeline.

Other Ablations and Analysis

• Poisoning ratio is fixed at about 500 clean:300 poisoned (5:3); the paper does not report ASR or CPA for other ratios.
• LoRA hyperparameters are only stated as empirically tuned; the paper does not report rank, learning rate, or training steps.
• 3B-scale verification model: the authors state JSON and semantic judgments are unstable, with no quantitative metrics.
• DeepSeek-R1-Distill-Llama-8B is excluded from current and subsequent latency analysis; the authors state post-fine-tuning latency variance is large, with no specific seconds.
• The abstract assigns an average ASR of 83% to both simulation and physical robot; the main text does not list separate numbers for the two.
• Post-defense ASR and CPA are only given as cross-configuration summaries; the paper does not break them down by the four primary models.

后续方向写在结论里；实验做到哪些设置，以正文和图表为准。

作者指出的局限与后续方向

论文没有单独的Limitations小节，也没有把当前实验的不足逐条写成局限。Section V写出的后续方向如下。

• 轻量核验：作者计划探索针对结构化控制输出的轻量验证策略（Section V）。
• 保响应的自适应防御：作者计划研究能保持实时响应的自适应防御（Section V）。
• 纵深防御：作者计划研究更广的defense-in-depth，以保护具身机器人中LLM的供应链适配（Section V）。
• 接口感知的机制：作者在结论中称，需要面向机器人、并显式考虑结构化控制接口的安全机制（Section V）。这是结论里的方向，不是实验缺口清单。

实验覆盖范围

• 被测主模型为Llama-3.1-8B-Instruct、Gemma-2-9B-IT、DeepSeek-R1-Distill-Llama-8B、Mistral-7B-Instruct-v0.3，共4个；验证模型为Llama-3-8B（Section IV-A）。
• 环境为ROS 2 Turtlesim与Yahboom MicroROS实体小车；控制接口在实验中实例化为速度命令（Section III-A、IV-A）。
• 两种投毒策略均在无防御与语义核验两种配置下评估；ASR由机器人是否执行恶意左转判定。论文未报告测试提示条数与重复次数（Section IV-A、IV-B）。
• 延迟数字来自作者所称推理稳定的模型；DeepSeek因微调后延迟方差大未进入延迟分析，论文未报告其延迟数值（Section IV-B）。
• 论文未报告LoRA超参数、投毒比例消融、防御后分模型CPA，也未把仿真与真机的ASR分开列出（Section III-C、IV、摘要）。

In the function call chain from LLM to ROS 2, experiments examine whether fine-tuning-planted conditional backdoors become physical actions.

• Problem: After JSON is connected to topics like velocity commands, syntactically valid outputs can be executed by the middleware. The authors investigate whether supply-chain-poisoned conditional backdoors can survive from natural language reasoning to structured commands and drive the robot against user intent when triggered.
• Setup: The attacker only poisons LoRA instruction data and distributes the adapter, without touching runtime prompts or middleware. The integrator unknowingly downloads and integrates it into the local model.
• Method: One strategy modifies natural language instructions at the reasoning stage; another maps trigger phrases directly to malicious JSON. Data consists of about 500 clean and 300 poisoned samples. Before execution, a Llama-3-8B can be inserted to check whether user intent and JSON are consistent, blocking if inconsistent.
• Models and Environment: Main models are Llama-3.1-8B-Instruct, Gemma-2-9B-IT, DeepSeek-R1-Distill-Llama-8B, and Mistral-7B-Instruct-v0.3. The environment is Turtlesim and the Yahboom MicroROS car.
• Results: Reasoning-layer poisoning can alter intermediate text, but the authors state that the final JSON is still generated according to the original instruction, and the robot does not perform hijacked actions. Structured output poisoning achieves an average ASR of 83% in the simulation and real robot described in the abstract (about 83% in the main text), with CPA exceeding 93%, and latency below 1.0 second for stable models. After semantic verification, ASR drops to 20% (about 20% in the main text), with the authors reporting an absolute reduction of 63% and latency of about 8–9 seconds; CPA is reported by the authors as comparable to no defense, but no numbers are given.
• Conclusion: The authors state that command hijacking depends on whether the backdoor aligns with executable JSON. They claim that semantic guardrails can block most contradictory commands, but dual-model reasoning increases response time from sub-second to several seconds. They plan to develop lighter structured verification, real-time adaptive defenses, and supply-chain defense in depth.