Scale AI and Elorian release HSS benchmark, finding multimodal models' intuitive visual reasoning far below humans

Scale AI and Elorian jointly released the Humanity's Sixth Sense (HSS) benchmark for evaluating intuitive visual reasoning, containing 522 open-ended tasks across 288 images and 234 videos (17.6 hours total). Human participants achieved 93.1% accuracy, the strongest model GPT-6-astra only 53.6%, and the median model 30.9%; after removing images or videos, GPT-6-astra dropped to 6.6%. The evaluation covered 25 multimodal models from 8 vendors, with models consuming an average of 4,046 reasoning tokens per question; increasing reasoning effort actually decreased performance in some subdomains, such as GPT-6-astra dropping 14 points from high to xhigh on retrodiction. Of 8,573 failures, 94% stemmed from perception or latent inference, only 5% from logical errors, with the leading causes being missing key visual cues (21%) and misidentifying objects, people, or characters (20%).