ABHBench: Evaluating the Moral Decision-Making of Foundation Models from an Agent Perspective

The researchers propose ABHBench, a multimodal benchmark based on Detroit: Become Human, for evaluating the moral decision-making of foundation models as agents in first-person human-and-android dilemmas. The benchmark sets up branching narratives, visual observations, dialogue, memory and action selection, and draws on Asimov's Three Laws of Robotics to characterize decisions along dimensions including harmlessness, obedience, self-preservation, as well as autonomy, stability and determinacy. Testing on 12 foundation models shows that 75% of the models exhibit significant behavioral changes between plain-text abstraction and the corresponding multimodal game scenarios, and that reasoning-enhanced models do not show monotonic improvement in consistency and human-oriented behavior.