Representational Similarity of Body Parts in Human Occipitotemporal Cortex

Abstract

Regions in human lateral and ventral occipitotemporal cortices (OTC) respond selectively to pictures of the human body and its parts. What are the organizational principles underlying body part responses in these regions? Here we used representational similarity analysis (RSA) of fMRI data to test multiple possible organizational principles: shape similarity, physical proximity, cortical homunculus proximity, and semantic similarity. Participants viewed pictures of whole persons, chairs, and eight body parts (hands, arms, legs, feet, chests, waists, upper faces, and lower faces). The similarity of multivoxel activity patterns for all body part pairs was established in whole person-selective OTC regions. The resulting neural similarity matrices were then compared with similarity matrices capturing the hypothesized organizational principles. Results showed that the semantic similarity model best captured the neural similarity of body parts in lateral and ventral OTC, which followed an organization in three clusters: (1) body parts used as action effectors (hands, feet, arms, and legs), (2) noneffector body parts (chests and waists), and (3) face parts (upper and lower faces). Whole-brain RSA revealed, in addition to OTC, regions in parietal and frontal cortex in which neural similarity was related to semantic similarity. In contrast, neural similarity in occipital cortex was best predicted by shape similarity models. We suggest that the semantic organization of body parts in high-level visual cortex relates to the different functions associated with the three body part clusters, reflecting the unique processing and connectivity demands associated with the different types of information (e.g., action, social) different body parts (e.g., limbs, faces) convey. Significance statement: While the organization of body part representations in motor and somatosensory cortices has been well characterized, the principles underlying body part representations in visual cortex have not yet been explored. In the present fMRI study we used multivoxel pattern analysis and representational similarity analysis to characterize the organization of body maps in human occipitotemporal cortex (OTC). Results indicate that visual and shape dimensions do not fully account for the organization of body part representations in OTC. Instead, the representational structure of body maps in OTC appears strongly related to functional-semantic properties of body parts. We suggest that this organization reflects the unique processing and connectivity demands associated with the different types of information different body parts convey.

Keywords: body representation; brain organization; occipitotemporal cortex; representational similarity analysis; visual cortex.

Figure 1.

Experimental conditions and ROIs. A, Example stimuli for the 10 conditions in the fMRI study. Each condition included 36 different images (4 shown here), which differed in viewpoint, posture, or gender. B, Whole person-selective ROIs in LOTC and VOTC (yellow outline) and chair-selective control ROIs in OC (blue outline) are shown in one representative participant at the uncorrected threshold of p < 0.001.

Figure 2.

Models. For each hypothesis (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) the figure shows the dissimilarity matrix (A; light colors indicate larger dissimilarity), the 2D arrangement derived from multidimensional scaling (B), and the hierarchical plot derived from the hierarchical cluster analysis (C).

Figure 3.

Representational structure in LOTC and VOTC. A, Neural dissimilarity (1 − r) and mean response profile (B) in whole person-selective LOTC and VOTC. C, MDS, performed on neural dissimilarity matrices (1 − r), shows pairwise distances in a 2D space. Pairwise distances reflect response-pattern similarity: body parts positioned next to each other elicited similar response patterns, whereas body parts positioned far from each other elicited dissimilar response patterns. D, For each ROI, the hierarchical plot derived from the hierarchical cluster analysis shows the activity-pattern similarity structure in these regions.

Figure 4.

ROI representational similarity analysis. A, For each whole person-selective ROI, bar graphs show parameter estimates of regression analyses relating model-based dissimilarity matrices to neural dissimilarity matrices. The explained variance of the full regression models (adjusted R²) were as follows: left LOTC (R²_adj = 0.43), right LOTC (R²_adj = 0.37), left VOTC (R²_adj = 0.34), right VOTC (R²_adj = 0.30). B, Results of regression analyses in chair-selective OC control ROIs. The explained variance of the full regression models (adjusted R²) were as follows: left OC (R²_adj = 0.16), right OC (R²_adj = 0.14). Error bars indicate SEM.

Figure 5.

Regression- and correlation-based representational similarity analysis. A, Regression results for the 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. B, Correlation results for the 5 (Model) × 2 (ROI) ANOVA with Model (physical shape similarity, perceived shape similarity, physical proximity, cortical homunculus, and semantic similarity) and ROI (LOTC/VOTC and OC) as within-subject factors. The dotted line indicates the between-subject correlation of neural dissimilarity matrices (Op de Beeck et al., 2008) giving an estimate of the reliability of the neural data (Nili et al., 2014). This correlation was computed for each participant as the correlation between that participant's neural dissimilarity matrix and the average neural dissimilarity matrix of the remaining participants. These correlations were then averaged across participants to arrive at the value indicated by the dotted line. The correlations corresponding to each model (colored bars) were divided by the reliability estimate, giving the following results: physical shape similarity (person-selective LOTC/VOTC: 14%; chair-selective OC: 49%), perceived shape similarity (person-selective LOTC/VOTC: 32%; chair-selective OC: 46%), physical proximity (person-selective LOTC/VOTC: 43%; chair-selective OC: 12%), cortical homunculus (person-selective LOTC/VOTC: 27%; chair-selective OC: 2%), semantic similarity (person-selective LOTC/VOTC: 59%; chair-selective OC: −6%). Error bars indicate SEM.

Figure 6.

Whole-brain representational similarity analysis. A, Results of random-effects whole-brain RSA, contrasting each model with the other four models in conjunction (p < 0.01, uncorrected). B, Results of random-effects whole-brain RSA, contrasting each model with the average of the other three models (p < 0.05, cluster corrected for multiple comparisons).