SocialGrid: A benchmark for planning and social reasoning in embodied multi-agent systems

The paper proposes SocialGrid, an embodied multi-agent environment inspired by Among Us, for evaluating the planning, task execution and social reasoning abilities of LLM agents. In testing, the strongest open-source model, GPT-OSS-120B, achieved below 60% accuracy on task completion and planning, and agents would fall into repetitive behavior or fail to get past basic obstacles. SocialGrid provides an optional Planning Oracle to evaluate social reasoning separately from planning deficits; even when planning assistance improves task completion rates, social reasoning remains a bottleneck, and agents of all sizes identify deception at close to random levels, relying on shallow heuristics rather than accumulating behavioral evidence. The environment also provides automated failure analysis and fine-grained metrics, and builds a leaderboard using Elo ratings from an adversarial league.