Study identifies seven measurement artifacts that fabricate language model progress in self-training

A new research paper published on arXiv by Zheng Xu, Nan Yan, Liming Chen, and Mohammad Taher Kashani reveals that evaluating the ability of language models to self-improve now relies on tracking individual problems gained or lost by the model, rather than relying solely on aggregate accuracy. The study notes that tracking these transitions involves calculating the difference between two noisy statistical estimates, leaving the results vulnerable to severe measurement artifacts that lead to misleading conclusions regarding the efficacy of self-training.

The researchers conducted an audit spanning three rounds of self-training using rank-32 Low-Rank Adaptation on a Qwen3-8B model, comparing the results to a frozen control run that underwent the exact same steps.The audit identified seven fundamental measurement failures, each capable of completely reversing the reported outcome in the absence of a measured control.The study showed that some of these flawed practices are standard conventions in AI research, where relying on an evaluation log built on a single greedy decoding creates fictitious capability shifts in a untrained model due to batching artifacts during inference, while the expansion metric distinguishing skill acquisition from refinement assigns a untrained model a score of 0.280.

The paper explains that remediation attempts based on conventional thresholding fail upon replication, as the null hypothesis estimated through frozen comparisons remains above zero. To overcome this obstacle, the team replaced those metrics with a rigorous per-problem statistical test against an aggregated baseline with false discovery rate control, which detected no phantom gains on held-out replicate samples and maintained its accuracy across multiple testing regimes, error rates, and batch sizes.

Applying the audit across matched runs showed that external distillation successfully solves problems rarely reachable by the base model, whereas three distinct variants of self-training fail to do so.The statistical analysis ruled out that this disparity was merely a byproduct of distillation's larger overall gains, with the p-value reaching decisive significance below 10 to the power of minus 8. While evidence remained inconclusive regarding problems completely unreachable by the base model, self-training was shown to degrade problems the model had solved successfully at baseline, at rates well exceeding the measured noise floor.

The study concludes that auditing shifts in model capabilities requires measuring an independent null hypothesis for every reported statistic, emphasizing that these null hypotheses do not require costly new experiments, but can instead be constructed directly from baseline replicates already available in multi-run studies.