A neurosemantic theory of concrete noun representation based on the underlying brain codes

Abstract

This article describes the discovery of a set of biologically-driven semantic dimensions underlying the neural representation of concrete nouns, and then demonstrates how a resulting theory of noun representation can be used to identify simple thoughts through their fMRI patterns. We use factor analysis of fMRI brain imaging data to reveal the biological representation of individual concrete nouns like apple, in the absence of any pictorial stimuli. From this analysis emerge three main semantic factors underpinning the neural representation of nouns naming physical objects, which we label manipulation, shelter, and eating. Each factor is neurally represented in 3-4 different brain locations that correspond to a cortical network that co-activates in non-linguistic tasks, such as tool use pantomime for the manipulation factor. Several converging methods, such as the use of behavioral ratings of word meaning and text corpus characteristics, provide independent evidence of the centrality of these factors to the representations. The factors are then used with machine learning classifier techniques to show that the fMRI-measured brain representation of an individual concrete noun like apple can be identified with good accuracy from among 60 candidate words, using only the fMRI activity in the 16 locations associated with these factors. To further demonstrate the generativity of the proposed account, a theory-based model is developed to predict the brain activation patterns for words to which the algorithm has not been previously exposed. The methods, findings, and theory constitute a new approach of using brain activity for understanding how object concepts are represented in the mind.

Figure 1. Locations of the voxel clusters (spheres) associated with the four factors.

The spheres (shown as surface projections) are centered at the cluster centroid, with a radius equal to the mean radial dispersion of the cluster voxels.

Figure 2. Correlation between LSA scores and activation-derived factor scores for the 60 words.

For the word length factor, the abscissa indicates the actual word length.

Figure 3. Correlation between independent ratings of the words and activation-derived factor scores for the 60 words.

For the word length factor, the abscissa indicates the actual word length.

Figure 4. Rank accuracy of identifying the 60 individual words for each participant and the group mean.

The accuracies are based on either the participant's own training set data (black) or on the data from the other 10 participants (gray), using factor-based feature selection (80 voxels) and the Gaussian Naïve Bayes classifier. The dashed lines indicate levels with p<.001 greater than chance, obtained with random permutation testing (black, within participants; gray, between participants).

Figure 5. Observed and predicted images of apartment and carrot for one of the participants.

A single coronal slice at MNI coordinate y = 46 mm is shown. Dark and light and blue ellipses indicate L PPA and R Precuneus shelter factor locations respectively. Note that both the observed and predicted images of apartment have high activation levels in both locations. By contrast, both the observed and predicted images of carrot have low activation levels in these locations.