Scapo readjusts reasoning rewards, semi-realistic stability untangles false correlations in model training

New research in reinforcement learning with verifiable rewards reveals that large language models, despite excelling at solving complex problems, remain highly sensitive to minor changes in command phrasing that are unrelated to the core of the problem. This weakness prompted a research team that includes Junshuo Pan and seven other researchers to submit a paper on arXiv, in which they analyze how token sensitivity varies within reasoning trajectories and propose an innovative method for adjusting reward weights during training to untangle superficial correlations that hinder model generalization.

Collective relative policy optimization algorithms award the final result equally to each token in the answer.This inadvertently reinforces random or fragile reasoning steps that only coincided with arriving at the correct solution. The researchers addressed this dilemma by testing semi-realistic interventions in the prompts, which are modifications that preserve the core problem and its original solution while altering the peripheral wording. The analysis showed that suppressing tokens with high deviation rates during the decoding stage alone contributes to higher reasoning accuracy without adjusting model weights.

Based on this causal observation, the team developed the Scapo algorithm as a modified version of collective policy optimization, integrating a semi-realistic stability criterion into the token-level reward distribution mechanism. The algorithm measures how much each token’s probability deviates under experimental modifications and uses standardized stability scores to reduce the gains of unstable tokens in the early phases of training, without granting any additional credit merely for stability. When the methodology was applied to two versions of the Qwen-3 model, with 4 billion and 1.7 billion parameters respectively, accuracy on the American Mathematics Competition tests for the years 2024-2026 increased by roughly 5.63 and 4.17 percentage points compared with the standard algorithm, achieving the best results on most mathematics tests and on all out-of-distribution generalization tests.

We see in this technical development a tangible shift that directly affects engineers and AI labs in the Gulf, Egypt and the Levant, particularly teams that train open-source models for financial analysis, legal consulting and programming tasks. The common reliance on rewarding the final result entangles development teams in models that appear superior in closed-lab experiments but become erratic when the tone or phrasing of real-world user queries changes. Adopting token-stability criteria enables developers in local work environments to improve the robustness of integrated and small models, thereby reducing the wasted compute cost of repeatedly training fragile-reasoning models.

Accounting for each reasoning step individually rather than relying solely on evaluating the overall result is the genuine path toward the maturation of AI agents.With the researchers’ public release of the source code, model-engineering teams can assimilate the logic of the Scapo algorithm and apply it to open-source model weights, providing software engineers or AI researchers with a practical tool to boost generative reasoning efficiency and reduce reliance on random memorization inference.