Benchmark Evaluation of Obstacle-Conditional Safety in Home Robot Navigation

The research proposes the OSMA-Bench benchmark and the Semantic Sensitivity Index (SSI) to evaluate whether home robot navigation policies can adjust motion parameters such as spacing and speed according to the type and distance of obstacles. The benchmark consists of sets of spatially matched scenarios that differ only in obstacle type, and SSI measures whether spacing and the speed and smoothness that vary with distance follow the caution ranking of obstacles. Evaluation of VLM-based and VLA-based policies shows that even with explicit safety prompts, policies cannot stably adjust motion according to obstacle safety; policies can correctly distinguish the safety relevance of obstacles, yet fail to reflect this distinction in the generated motion parameters or actions. For VLM policies that first identify obstacles and then generate navigation programs, adding an explicit parameter inference step between the two stages improves adaptation. The authors state that these findings expose a gap in semantic-to-motion grounding.