Nine AI models fail to outperform the market in Quant Arena
A recent research study led by researcher Hudi Kian and 11 co-authors showed that agents of large language models are still unable to achieve genuine outperformance in financial markets when tested in realistic trading environments. The researchers launched an evaluation platform and a public leaderboard called "Quant Arena" to test the models’ ability to complete an entire quantitative research cycle on real data from the Chinese stock market. After testing nine leading models, headed by the "GPT-6 Astra" model, it turned out that none of them managed to outperform the equal-weight market performance during 2024.
The paper starts from observing a fundamental flaw in previous metrics for evaluating AI in the finance sector; they had been limited to measuring the accuracy of writing mathematical formulas, checking code quality, or assessing isolated individual trading decisions. This partial evaluation ignored the most important challenge, which is completing the iterative cycle undertaken by the financial researcher: formulating a hypothesis, turning it into an executable signal, testing it on historical data, and then reporting what is worth relying on. The new platform was introduced to test the models’ ability to produce real trading strategies that hold up outside the research time horizon.
The evaluation system on the platform uses a strict data-isolation mechanism; the model builds programming features and stock-classification code and receives feedback only on data from 2022 and 2023. After the cycle is completed, a single strategy for each model is frozen without any further modification, to be subjected to a final test on fully closed 2024 data. To avoid the trap of theoretical numbers, the cash-account evaluation system applied realistic trading constraints and costs, including measuring the impact of trades on prices through order-book simulation, with the final score calculated based on the incremental return achieved, the quality of algorithmic signals, and the diversity of features used.
When we examined the test results, a structural gap among developers became apparent; the ability to craft complex programming code and statistical features during the 2022-2023 window did not translate into real financial performance in the unseen 2024. Although "GPT-6 Astra" topped the overall ranking, all models failed to keep pace with the simple market index once transaction costs and trading impact were applied, revealing a tendency of smart agents to over-fit historical data and produce strategies that look profitable on test screens but collapse in the face of actual market structural changes.
What actually changes in trading decisions and asset management in the region?This research puts quantitative trading desks and investment funds in Riyadh, Abu Dhabi, Dubai and Cairo on an executive warning bell: writing code and generating financial hypotheses with AI models does not mean possessing a strategy capable of capturing profits. If you work in regional asset management and test automation models on stocks such as "Tadawul" or the Abu Dhabi market, relying on fast historical test results without simulating price slippage and liquidity depth will inevitably lead you to uncalculated losses.
This reality forces financial teams in the Middle East to redirect their efforts and budgets toward building rigorous audit and simulation platforms that enforce order-book constraints and data isolation, rather than merely showcasing the models’ ability to write code equations. The real bet in the region’s markets is not merely having tools that generate investment strategies, but the ability to filter them and test their resilience against real market frictions before risking a single dirham of capital.