Identifying bilingual semantic neural representations across languages

Abstract

The goal of the study was to identify the neural representation of a noun's meaning in one language based on the neural representation of that same noun in another language. Machine learning methods were used to train classifiers to identify which individual noun bilingual participants were thinking about in one language based solely on their brain activation in the other language. The study shows reliable (p<.05) pattern-based classification accuracies for the classification of brain activity for nouns across languages. It also shows that the stable voxels used to classify the brain activation were located in areas associated with encoding information about semantic dimensions of the words in the study. The identification of the semantic trace of individual nouns from the pattern of cortical activity demonstrates the existence of a multi-voxel pattern of activation across the cortex for a single noun common to both languages in bilinguals.

Fig. 1

Schematic representation of the experimental paradigm for the (A) English and (B) Portuguese acquisitions.

Fig. 2

Cross-language identification of neural representations for nouns: identification accuracies for the classifier trained on the brain activation for words for one participant used to predict the same words in the other language in the same participant. Classification for Eng–Pt is M = .68; SD = 0.11; and for Pt–Eng is M = .72; SD = 0.08. Reliable accuracies (rank-accuracy > .63; p < .05) were obtained for seven participants in English-to-Portuguese (Eng–Pt) cross-language classification, and for 10 participants in Portuguese-to-English (Pt–Eng) classification. Results are rank-ordered from highest to lowest classification rank accuracies for the Pt–Eng classification (white bars).

Fig. 3

Union of stable voxels for English words and for Portuguese words for all participants. Stable voxel clusters (minimum five stable voxels) show similarities in brain areas where the voxels used for classification of brain activation across languages were located. Yellow ellipses highlight areas in the language network (LIFG and left posterior superior temporal lobe). Red ellipses highlight areas in the postcentral gyrus. Green ellipses highlight areas in the left inferior parietal sulcus.

Fig. 4

Voxels used in pattern-based classification that encode the semantic representation of object manipulation and shelter in both Portuguese and English shown in red or yellow. Figure shows voxel clusters from the union of stable voxels across participants. Green circles highlight one of the manipulation factor areas in left supramarginal/postcentral gyrus reported by Just et al. (2010) (centroids are x = −60, y = −30, z = 34; 51 voxels and a radius of 10.0 mm). Blue circles highlight one of the shelter factor areas in left fusiform/parahippocampal gyri reported by Just et al. (2010) (x = −32, y = −42, z = −18; 26 voxels and a radius of 6.0 mm). Yellow areas indicate stable voxels in the current study.

Fig. 5

Common areas of brain activation for thinking about the properties of tools and dwellings in English and Portuguese. Group-level contrast for Dwellings > Fixation (left) and for Tools > Fixation (right) for English (red), for Portuguese (blue), and for the commonalities between the two languages (green); p < .001 uncorrected; T = 4.14; extent threshold = 6 voxels.