Import AI 476: visual reasoning benchmarks and AI research taste
Start here
Elorian and Scale AI released the Humanity's Sixth Sense (HSS) benchmark, testing intuitive visual reasoning in multimodal models, containing 522 questions, 288 images, and 234 short videos. Human pass@1 is 93.1, while the highest-scoring GPT 6 Astra only reaches 53.6; Chinese open-weight models GLM-5.3-flash and Kimi K3 score only 26.8 and 25.5, respectively. P-Zero Research's TasteVal benchmark evaluates AI research taste using 8 tasks.
Generated by AI from reports · Updated 12 hr ago
Progress timeline
- Import AIImport AI 476: creativity and RSI, open-weight models falling behind China, AI heading to space
Elorian and Scale AI release the Humanity's Sixth Sense (HSS) benchmark, testing multimodal models' intuitive visual reasoning: human pass@1 is 93.1, while the top-scoring GPT 6 Astra reaches only 53.6. The benchmark contains 522 questions, 288 images, and 234 short videos; Chinese open-weight models GLM-5.3-flash and Kimi K3 score only 26.8 and 25.5, respectively. P-Zero Research's TasteVal benchmark uses 8 tasks to evaluate models' "research taste," while AI Frontiers discusses the compounding advantages of AI expanding into space.
Related discussion
Attention trend
New sources per day
- Oct 12New1 sources (1 outlets,0 accounts)
- Oct 13New0 sources (0 outlets,0 accounts)