Multivariate Pattern Analysis Reveals Category-Related Organization of Semantic Representations in Anterior Temporal Cortex

Abstract

The neural substrates of semantic representation have been the subject of much controversy. The study of semantic representations is complicated by difficulty in disentangling perceptual and semantic influences on neural activity, as well as in identifying stimulus-driven, "bottom-up" semantic selectivity unconfounded by top-down task-related modulations. To address these challenges, we trained human subjects to associate pseudowords (TPWs) with various animal and tool categories. To decode semantic representations of these TPWs, we used multivariate pattern classification of fMRI data acquired while subjects performed a semantic oddball detection task. Crucially, the classifier was trained and tested on disjoint sets of TPWs, so that the classifier had to use the semantic information from the training set to correctly classify the test set. Animal and tool TPWs were successfully decoded based on fMRI activity in spatially distinct subregions of the left medial anterior temporal lobe (LATL). In addition, tools (but not animals) were successfully decoded from activity in the left inferior parietal lobule. The tool-selective LATL subregion showed greater functional connectivity with left inferior parietal lobule and ventral premotor cortex, indicating that each LATL subregion exhibits distinct patterns of connectivity. Our findings demonstrate category-selective organization of semantic representations in LATL into spatially distinct subregions, continuing the lateral-medial segregation of activation in posterior temporal cortex previously observed in response to images of animals and tools, respectively. Together, our results provide evidence for segregation of processing hierarchies for different classes of objects and the existence of multiple, category-specific semantic networks in the brain.

Significance statement: The location and specificity of semantic representations in the brain are still widely debated. We trained human participants to associate specific pseudowords with various animal and tool categories, and used multivariate pattern classification of fMRI data to decode the semantic representations of the trained pseudowords. We found that: (1) animal and tool information was organized in category-selective subregions of medial left anterior temporal lobe (LATL); (2) tools, but not animals, were encoded in left inferior parietal lobe; and (3) LATL subregions exhibited distinct patterns of functional connectivity with category-related regions across cortex. Our findings suggest that semantic knowledge in LATL is organized in category-related subregions, providing evidence for the existence of multiple, category-specific semantic representations in the brain.

Keywords: anterior temporal lobe; language; learning; multivariate pattern analysis; reading; semantic memory.

Figure 1.

Pseudoword training performance and semantic priming results. A, Mean accuracies and RTs for identification of pseudoword category across training sessions. B, RTs in the priming experiment completed after the eighth training session. Bars represent mean RT for the different priming conditions: DWSC, prime was different word, same subordinate category as target; DWDC, prime was different word, belonging to different subordinate category than target; DWDSC, different word, different superordinate category than target. Plot is grouped by category of the target word (animal or tool). Brackets indicate significant differences across priming conditions: *p < 0.05. Errors bars indicate SEM.

Figure 2.

fMRI oddball task paradigm and MVPA analysis schematic. A, Subjects performed an oddball detection task in the scanner, in which they responded with a button press whenever a TPW for an oddball animal or tool category was presented (illustrated here by the green frame, which was not visible in the experiment). B, Classifiers were trained on β images estimated from five TPWs for each animal and tool category. The classifiers were then tested on β images generated from a separate set of five TPWs for each category.

Figure 3.

Superordinate and basic level classification of TPW semantics in the LATL. A, Significantly above chance decoding of TPW superordinate category in LATL (MNI peak coordinates: −38, −4, −26). Color bar represents t statistic. Thresholded at a voxelwise p < 0.005 and cluster-level p < 0.05 (FWE-corrected). B, Basic level classification of animals (blue; peak coordinates: −48, −12, − 20) and tools (red; coordinates: −36, −18, −28) in distinct subregions of LATL. Thresholded at a voxelwise p < 0.005 and cluster-level p < 0.05 (FWE-corrected). For visualization purposes, other significant ROI (LIPL and rSFG, shown in Fig. 4) have been masked here.

Figure 4.

Superordinate and basic level classification of TPW semantics in LIPL and rSFG. A, Classification of TPW superordinate category in LIPL (peak coordinates: −24, −44, 42). B, Basic level classification of tool TPW in LIPL (coordinates: −36, −64, 44). C, Basic level decoding of animal TPW in rSFG (coordinates: 10, −4, 56). Color bar represents t statistic. Thresholded at a voxelwise p < 0.005 and cluster-level p < 0.05 (FWE-corrected).

Figure 5.

Functional connectivity of tool- and animal-selective subregions of LATL. A, Connectivity of the tool-selective LATL subregion. B, Connectivity of the animal-selective LATL subregion. C, The tool-selective subregion showed greater connectivity with left ventral premotor cortex and LIPL (but not right, data not shown) (orange/red), and the animal-selective subregion showed greater connectivity with bilateral middle temporal gyri (green/blue). Color bar represents t statistic. Thresholded at a voxelwise p < 0.005 (uncorrected) and cluster-level p < 0.05 (FWE-corrected).