Controlling knowledge distillation mitigates visual models’ tendency to beautify distorted text

Probabilistic vision-and-language models tend to correct anomalous or unfamiliar text within images and turn it into linguistically acceptable expressions, a behavior that harms literal fidelity in text extraction and optical character recognition tasks. When the model encounters a unfamiliar word or a unusual operational number, its pre-training pushes it to replace the recorded reality with the most likely form in context, causing unintended distortion of documents. To address this challenge, researchers Baodi, Wang, Zomming Huang, Kixuan Ren, John Huang and Wei Zhu introduced a new methodology called “GAD-RL,” which relies on direct knowledge distillation organized through reinforcement learning to protect transmission accuracy.

The study begins with a precise observation of training mechanisms, noting that comprehensive task rewards at the sequence level and local teacher guidance theoretically complement each other, but the supervision of a fixed teacher gradually loses its value as the student’s model level advances. The disconnectedThe study begins with a precise observation of training mechanisms, noting that comprehensive task rewards at the sequence level and local teacher guidance theoretically complement each other, but the supervision of a fixed teacher gradually loses its value as the student’s model level advances. The disconnected experimental analysis showed that continuous training under a fixed teacher becomes a burden as training checkpoint scores increase, and its negative effect varies across different response groups, since requiring the student to follow the teacher in cases of advancement hinders their independence and limits the accuracy of their decisions when dealing with rare cases.

The researchers devised an adaptive system that relies on a frozen teacher receiving the attribution texts and the beginnings generated by the student model, with precise control of training signals during the shared post-training stage.The methodology completely disables distillation for response groups that achieve a reward of 0.95 or higher, while gradually reducing the strength of distillation as the group's average reward increases.At the same time, the weighting of the front “feedback-labeler” gap is recomputed based on the probability that the student selects the higher-level symbol from the teacher, limiting further local adjustments if the student's support for the candidate is weak and protecting the model from noise.

The mechanism was applied to the integrated “Qwen 3.5” model with 2 billion parameters and yielded benchmark results that reflected the robustness of the design. The model achieved a precise retrieval accuracy of 59.92 % on the Chaos Bench standard for anomalous and complex texts, surpassing the reinforcement-learning-with-human-feedback algorithm by 8.45 percentage points and out-performing the static combination of reinforcement learning and distillation by 4.43 percentage points. It also scored 91.18 on the OmniDoc Bench benchmark, confirming its ability to maintain literal accuracy and read intertwined documents without falling into linguistic guessing traps.

This pathway represents a qualitative shift for development teams, banks, and government institutions in the Gulf and Egypt, where millions of official documents, contracts, invoices, and identity records undergo digital processing daily. An error in reading a tax code or correcting a rare serial number represents a significant operational and legal risk that financial systems cannot bear, and this technology enables the building of small, efficient models of two billion parameters that operate locally, safely, and with limited computing cost, without needing to rely on expensive external inference platforms or replacing written truth with probabilistic predictions.

Integrity in reading documents requires restraining models' ability to guess, and protecting the original text from algorithms' desire to correct it.Adaptive training gives tech companies in the region a practical way to acquire integrated, reliable document-processing engines that ensure the system reads exactly what was originally written, putting an end to beautifying text at the expense of data integrity.