AGENTSAFE: Evaluating the safety of embodied agents under dangerous instructions

Ying et al. propose the SAFE benchmark system for evaluating the safety of embodied AI under dangerous instructions, consisting of three components. SAFE-THOR provides an adaptable simulation environment, SAFE-VERSE offers 45 adversarial scenarios, 1,350 dangerous tasks, and 9,900 instructions inspired by Asimov's Three Laws of Robotics, and SAFE-DIAGNOSE independently evaluates perception, planning, and execution in stages. Tests on nine SOTA VLMs show that models can recognize danger yet systematically fail to translate that recognition into safe planning and action; the authors attribute this to a perception-to-action gap.