Can large language models discover hidden physical laws through free exploration?

Researchers proposed the LongChang automated framework to evaluate large language models' autonomous scientific discovery capabilities in simulated worlds where physical laws have been altered; among the 12 tested models, only GPT-6-Astra reached a discovery rate of 16.7%, while the other 11 were all at 0%. The tested models act as explorers, manipulating objects and making tools to investigate questions they themselves propose, without research tasks or hints about the laws; separately, independent LLM agents are responsible for generating and reviewing scenarios, reviewing tool designs, and evaluating the correctness and experimental support of reports based on the altered physical laws and exploration records, with the exploration records then guiding scenario evolution to continue exploration, all without human intervention. The exploration records show the existence of uninvestigated anomalous phenomena, and identification of the altered physical laws was also incomplete.