Chemical knowledge organizes in the middle layers of models before they utter it and exposes the shortcomings of output tests

When a small language model is asked a direct chemical question about the basic functional group in 1-fluorobutane, it may suddenly answer that it is an alkyne group instead of an alkyl fluoride. At first glance, this failure seems to be evidence of the model's ignorance of molecular structure and its inability to detect the fluorine atom. However, a recent research study led by researcher Tran Nhat Khanh Nguyen with three other researchers, published on arXiv, reveals in the details of our examination that judging language models solely from their textual outputs falls into a misleading trap.

The paper traced the internal representation of five open-source models across prompts that included 92 molecules and 20 chemical functional groups, to answer a fundamental question: did the model fail to recognize the chemical element, or did it actually detect it but fail to express it?The scientific surprise was that the models, even before any specialized chemical training, organize functional groups internally into chemically coherent families and classes that are mathematically consistent.

Experiments show that the models encode chemical differences as matching directions in vector space. When mathematically moving from thioether to thiol and adding it to the ether, the vector settles near the alcohol with a precision that mirrors the actual chemical relationships. This organized structure unexpectedly emerges in the middle layers of the model, where internal representations suddenly shift from merely tracking the wording of the prompt to tracking the functional group and its abstract chemical properties.

This distinction between latent knowledge and verbal expression appears in the base model’s numbers, which did not exceed a correct functional-group naming rate of 58.5 % of the prompts, the lowest among the five tested models. Nevertheless, the model internally retained chemically meaningful relationships, and its errors were not incorrect answers but a failure to provide an answer. When the models were subjected to precise, chemistry-specific fine-tuning, the representations reorganized so that the chemical concept became dominant in the internal representation, outweighing the phrasing of the question.

Analyzing internal representations changes the evaluation criteria for R&D teams and materials labs in the region.Thus, when engineering teams in Gulf energy facilities or tech centers in Egypt, the Levant, and elsewhere build intelligent agents for synthesis planning and material suggestion, they often discard open-source models as soon as they see low accuracy rates in the outputs, opting instead to incur the cost of calling large commercial models. Yet recognizing that a coherent chemical engineering exists in the middle layers directs engineers to invest effort in probing those representations and extracting them rather than paying escalating inference bills.

Assessing artificial intelligence in the physical sciences should not be limited to the final word spoken by the model, but should extend to examining what has settled in its deep architecture. The ability to distinguish between missing information and difficulty expressing it gives developers of chemical systems in our technical environments a finer criterion for selecting and tuning models, turning models that appear weak on paper into pillars relied upon for automating scientific discovery.