Two AI Models Learn Together, Then Reason Alone

What if solving a difficult problem did not require running a team of models for every request? Research from Mohamed bin Zayed University of Artificial Intelligence, or MBZUAI, proposes moving collaboration to an earlier stage: two models exchange solution methods during training, then each answers independently afterwards. The idea does not promise to eliminate computing costs, but it attempts to shift some of them to training rather than repeating them with every question.

This late signal is worth noting: The university published details of the work on July 27, and the study was recently presented at the 43rd International Conference on Machine Learning in Seoul. The researchers named the framework Collaborative Reasoning via Cross Teaching, abbreviated to CoRe. The study is led by Kshitij Mishra, a postdoctoral associate in machine learning at the university, with Mirat Aubakirov, Martin Takáč, Nils Lukas and Salem Lahlou.

The team starts from a practical problem. When several models collaborate at inference time, when the answer is produced, costs rise because the system runs more than one model. Repeating the same model also does not guarantee that an error will be corrected. If the copies share the same blind spot, they may all reach the wrong conclusion. The researchers call this correlated failure, and say a weaker model is useful only when it is right where the stronger model is wrong.

Collaboration does not help when errors overlap: The team therefore designed the training process to reward diversity in reasoning paths. In the first round, both models try to solve the problem independently and without hints. If one succeeds, its reasoning becomes material that helps the other in a second round. The final answer is not included in the hint, so the model that struggled sees its peer’s steps without receiving a ready-made solution.

CoRe adds another measure that the researchers call hint dropout. In some cases the hint is withheld so the model does not always depend on it, because each model will work alone when deployed. The framework uses reinforcement learning, with a reward for a correct answer, a rescue bonus when a model corrects its first-round error, and an exploration reward when it tries a different path.

The team tested two pairs of small models, with each pair totalling no more than about seven billion parameters. The first paired Qwen2.5-3B-Instruct with Qwen3-4B-Instruct, while the second paired Phi-4-mini-reasoning with Ministral-3-3B-reasoning. The tests covered four benchmarks in mathematics, problem solving and scientific questions, using no more than 1,000 training examples for each benchmark.

The numbers are strong within the experiment’s limits: The Qwen pair’s performance on the GSM8K dataset rose from about 60% before collaborative training to 99.54% afterwards. The reasoning pair reached 92% on the MATH benchmark, about 80% on AIME, and more than 77% on GPQA, which covers graduate-level science questions. When the researchers tested the models individually, Phi-4-mini-reasoning’s accuracy on MATH rose from 57% to 85%.

These are research results, not a guarantee of product performance inside an institution. The tests are defined, the models are small, and the team itself leaves an open question: when is collaboration during training better than collaboration at the point of use? That caveat does not diminish the idea’s value, but defines the stage it has reached.

For the region, the issue is building knowledge rather than importing an answer: The lens here is economic transformation. A research institution in Abu Dhabi is not merely running an off-the-shelf model, but proposing a training method that may make smaller models more useful after learning. If the results hold across broader tasks, languages and environments, the approach could help institutions with limited computing budgets test a path based on improving training rather than adding models to every request.

The honest conclusion is that CoRe does not yet establish that two small models can replace larger systems in real work. What it demonstrates within the published experiments is narrower and more useful: exchanging solution methods during training can improve both the team’s performance and that of each model working alone. The next step to watch is not a bigger headline, but independent tests on Arabic tasks and in operational contexts that measure accuracy and cost together.