Beware the Untrusted Simulator — Reward-Free Backdoor Attacks in Reinforcement Learning

Researchers propose a reinforcement learning backdoor attack named Daze, which exploits the dynamics of an untrusted simulator to stealthily implant an action-level backdoor into RL agents trained for real-world tasks without modifying or observing the agent's reward. The attacker can reliably activate the target action when the agent observes a predefined trigger condition, leading to potentially dangerous consequences. The paper provides a formal proof that Daze guarantees attack success on general RL tasks, and conducts extensive empirical evaluations in discrete and continuous action space domains; the authors state this is the first case of an RL backdoor attack transferred to real robot hardware. The authors hope this will advance research on security defenses for all components of the RL training pipeline.