RoboJailBench: A Benchmark Evaluation of Adversarial Attacks and Defenses on Embodied Robotic Agents

The researchers propose RoboJailBench to standardize the evaluation of jailbreak attacks and defenses on embodied robotic agents. The benchmark includes a safety taxonomy built on ISO standards, regulatory rules and documented incidents, summarizing 18 categories of embodied AI safety violation consequences; its intent-contrast data pipeline supplements existing data with paired adversarial goals and benign goals, to measure both safety and the ability to follow benign instructions. On this basis the research team constructed a new class-balanced dataset and augmented five existing datasets, integrated four attacks and two defenses, evaluated on leading embodied VLMs, and released code, datasets and a leaderboard.