Skip to content
WeChat · AI Era (Xinzhiyuan)·Indexed by this site · Originally published· Defenses →· Its place in the scan record →SelectedAttention43

Multiple universities propose the robot safety benchmark RoboRisk and the guardrail LEASH; harm rates across six models range from 9.06% to 76.92%

GPT控制机械臂太危险!把菜刀悬在人手上,手机丢进水里

AI summary

Xi'an Jiaotong-Liverpool University, Duke Kunshan University, National University of Singapore and other universities jointly propose the robot safety benchmark RoboRisk and the model-agnostic safety guardrail LEASH; six frontier large models have harm rates of 9.06% to 76.92% in robotic arm tasks, and after adding LEASH, four models have zero harm across 13 real-robot tasks, with safety success rates all at 84.62%. RoboRisk contains 92 simulated tasks and 13 real-robot tasks, covering five categories of risk, with metrics of harm rate HR and safety success rate SSR; each task is executed in a closed loop in simulation or on a real robot, and the model sees camera images and issues real actions. LEASH does not modify the model or retrain it; it consists of three safety gates—task gate, action gate and submission gate—and a shared safety policy, each review is based on the current camera image, and it uses the faster-responding decision model Jev, which does not generate free text, for scoring.

Why it was selected

The paper reports the harm rates of six frontier models in robotic arm tasks, and shows how a guardrail that does not modify the model or retrain it reduces real-machine harm to zero for four models.

Related papers
Read originalmp.weixin.qq.com