Skip to content

NeurIPS 2026: Psychological tests on 14 models reveal an internal thinking gap

Share
NeurIPS 2026: Psychological tests on 14 models reveal an internal thinking gap

Listen to this article

Read by Anchor

Do large language models possess a truly internal representation of psychological structures, or do they merely generate responses that superficially mimic personality measures? This pivotal scientific question now has experimental evidence, after a paper accepted at the NeurIPS 2026 conference revealed standard psychological inference mechanisms within 14 variants from the Llama 3 and GPT-2.5 families, whose sizes ranged from half a billion to 14 billion parameters in their base and instruction-tuned versions.

Researchers Chieh-Hao Hsu, Feng-Chun Cho and Hai-Chun tested the models for multi-personality embodiment, using real responses from 272 university students in Taiwan across seven psychological scales. The research team gave the models a single psychological-scale score for each individual and asked them to predict the responses on the remaining six scales. The behavioral results showed that the larger models reproduced the mutual correlations among the human scales, but amplified them, and this behavioral amplification persisted even after the students’ data were corrected for measurement weakness.

The behavioral amplification was not merely accidental; it was directly linked to the strength of geometric alignment in the instruction-tuned model layers.

When the researchers examined the internal layers via contrastive direction analysis, they found that the internal geometry of the large instruction-tuned models aligned precisely with the matrix of human correlations, whereas this alignment was absent in the untuned base models. The measurement showed that the strength of geometric cohesion correlated positively with behavioral amplification, with a correlation coefficient of 0.68, remaining at 0.69 after neutralising the effect of logarithmic model size.

To understand how the internal structure translates to textual outputs, the team applied a matched activation probing pattern and discovered a striking disparity: activation-probe slopes fell in a narrow range between 1.38 and 1.77, compared with behavioral slopes ranging from 0.52 to 1.53. Through compositional control, they showed that a ridge probe amplifies those slopes, but after subtracting a baseline based on an alternative reliability measure, the adjusted slopes remained strongly linked to behavior, with a strength of 0.88. This gap between activation indicators and outputs led the researchers to coin the term “reading weakness,” while noting that observing geometric alignment does not necessarily prove that the measured pathways are the direct causal driver of the answers.

Simulating behavioral response does not necessarily mean the model possesses an organized cognitive representation guiding its decisions.

These findings demand a practical shift in decision-making mechanisms at institutions and banks in the Gulf and Egypt that integrate language models into psychological assessment of job candidates or the construction of virtual personalities for customer service. The models’ amplification of correlations and the emergence of a reading-weakness gap warn against relying solely on directive output, which can produce misleading and exaggerated impressions of virtual personality behavior. AI teams in the region should audit the models’ internal layers and examine activation indicators before deploying them in human-evaluation settings, rather than merely testing generative text on user interfaces.

Don't miss the next story

Subscribe for updates