Logging errors in agent context pushes small models to repeat the mistake instead of correcting it

Developers of autonomous systems have traditionally assumed that providing a language model with a record of its previous failure would steer it toward the correct behavior, but we followed, in the latest material deposited in the academic platform “Archive”, a study published by researcher Ismail Jam'a that shows this feedback leads to a completely opposite outcome in small models. Experiments conducted on hexagonal and board games with strict rules, such as “Sokoban”, “Connect Four” and “Atari”, demonstrated that logging the erroneous move and its failure message within the context actually increases the likelihood that the model repeats the same rejected step, achieving what the paper described as a negative corrective gain of 1.38 nat per token across eight instruction-tuned models whose sizes ranged from 135 million to 1.7 billion parameters.

This flaw, according to the paper’s exploratory analysis, originates from the superficial form of the move expression in the text rather than from logical understanding of the game rules; it appears that inserting the failed-step text makes the logarithm of its repetition probability almost independent of the model’s prior desire to play it, with a slope that does not exceed 0.18.Moves that were not among the model’s options suddenly become weighted in its calculations.which causes greedy decoding algorithms to actually produce them. Conversely, the steps that the model was about to execute see their theoretical probabilities drop, but decoding returns and repeats about 38 percent of them at execution time, a behavior that is clearly confined to illegal moves rather than legally losing moves, with its severity varying from model to model.

The paper tested common engineering solutions, and the experiment revealed that padding the context with an explicit instruction such as “do not repeat the step” provided no practical benefit, while the “clean retry” option performed worst across runtime environments. Likewise, blocking tokens at the decoder level only prevented literal text duplication without stopping the agent’s illegal behavior. In contrast, the study presented two decisive technical takeaways:Removing the failed-move text from the error message preserves the model’s original preferences.and prevents its regression; subsequent training scaling for the one-billion-parameter “Olmo 2” model showed that this inversion emerges with the base model and recedes substantially when it undergoes a direct-preference fine-tuning stage, whereas virtual “RAG-in” framework models exhibit this inversion in their own Sokoban format even before reinforcement learning is applied.

This finding touches the reality of engineering gaps and smart-agent developers in the region, particularly in the Gulf, Egypt and the Levant, where local projects and institutions are moving toward deploying lightweight, open-source models within their own environments to control compute costs and ensure data sovereignty. The prevailing reliance among developers on runtimes that repeatedly pass the full textual failure log exposes local automation pipelines to costly failures, especially when managing APIs or functions that require strict rule adherence. Correcting this trajectory requires your project to immediately stop passing corrupted textual commands in alert messages and to focus on advanced post-training frameworks such as direct-preference fine-tuning rather than betting on naïve textual directives.

The research paper stood out for its precise measurement methodology, executed with single-precision floating-point on a free GPU in the “Colab” platform after first successfully reproducing the reference results. In summary, our reading of the material concluded thatNaïve error handling within the runtime doubles the confusion of smart agents.instead of training them on the correct behavior, which requires every engineer building an independent agent to meticulously reengineer conversation logs before deploying them in real-world work environments.