Longchang test for altered physics: one model discovers the laws and beats 11 competitors
Scientific discovery usually starts from observing a unexpected phenomenon that prompts the researcher to ask a new question, but transferring this practice to language models still collides with a deep gap between text generation and understanding physical reality. In a recent study deposited by researchers on the arXiv platform, AI models were subjected to a test that measures their ability to detect anomalous behaviors in virtual environments whose physical laws have been altered, such as heavier objects falling more slowly, to see whether these models can uncover hidden laws through free experimentation without prior guidance specifying what they should look for.
To apply this test, the researchers developed an automated framework they called “Longchang”, which is based on simulating worlds in which one or more physical rules differ from ordinary reality. In these worlds, the tested models assumed the role of explorers who interact freely with objects and create custom experimental tools to answer their own questions, without receiving directed tasks or hints that reveal the nature of the altered laws. The experiment relied on a multi-agent system in which separate models handled scene construction and review, tool-design inspection, and evaluation of final reports to verify their accuracy and experimental support based on exploration logs, allowing the full research cycle to be completed without any human intervention.
When we examined the details of the results, the numbers revealed a sharp disparity exposing the limits of the models’ actual independence; of the twelve models subjected to the experiment, only “GPT-6 Astra” achieved a discovery rate that did not exceed 16.7 percent, while the other eleven models remained at zero percent.A review of the exploration logs showed a notable methodological flaw, as the models observed anomalous phenomena within the simulation environment but left them without investigation or follow-up, and were satisfied with a partial observation of the altered physical laws without constructing a scientific inference that links premises to outcomes.
This failure redraws the stakes for those working in technology firms and innovation and digital transformation centers in the Gulf, Egypt and the Levant, where energy, logistics and industrial-facility sectors in the region are moving toward expanding digital twins and assigning smart agents to monitor simulations of complex systems. If you oversee the development of these systems, relying on models to conduct free exploration or to detect operational and physical deviations without precise task definition remains a uncalculated risk. The skill required today from engineering teams is not to unleash agents on their own, but to engineer strict frameworks that obligate the model to track any data anomaly and analyze it, while keeping human review as an indispensable safety valve.
The experiment demonstrates that the ability to generate logical answers does not necessarily imply scientific curiosity that leads to uncovering the unknown.Although a single model achieving a modest percentage in decoding the laws represents the start of a long path, the inability of the remaining models to move beyond zero confirms that the shift from language simulation to real-world mechanical understanding requires a comprehensive review of how independent agents are built and how they interact with the surrounding physical reality.