Aligning transformer circuit mechanisms to neural representations in relational reasoning

Relational reasoning-the capacity to understand how elements relate to one another-is a defining feature of human intelligence, yet its computational basis remains unclear. Here, we combined human neuroimaging (7T fMRI) and artificial neural network modeling to examine relational reasoning in biological and artificial systems. Using the Latin Square Task, we found that humans and transformers were able to generalise the task reliably, while standard architectures used in cognitive neuroscience could not. Analysing the transformer components revealed distinct computational roles: positional encoding captured the spatial structure of the task and aligned with representations in visual cortex, whereas attention encoded relational structure and mapped onto frontoparietal and default-mode networks. Attention weights tracked the relational complexity of the task, providing a computational analogue of working-memory demands. These results demonstrate convergent computational strategies for reasoning in brains and transformers, highlighting attention-based architectures as powerful models for investigating the neural basis of higher cognition.