Body and object effectors: the organization of object representations in high-level visual cortex reflects body-object interactions

Abstract

The principles driving the functional organization of object representations in high-level visual cortex are not yet fully understood. In four human fMRI experiments, we provide evidence that the organization of high-level visual cortex partly reflects the degree to which objects are typically controlled by the body to interact with the world, thereby extending the body's boundaries. Univariate whole-brain analysis showed an overlap between responses to body effectors (e.g., hands, feet, and limbs) and object effectors (e.g., hammers, combs, and tennis rackets) in lateral occipitotemporal cortex (LOTC) and parietal cortex. Region of interest analyses showed that a hand-selective region in left LOTC responded selectively to object effectors relative to a range of noneffector object control conditions (e.g., graspable objects, "act-on" objects, musical instruments). Object ratings showed that the strong response to object effectors in hand-selective LOTC was not due to general action-related object properties shared with these control conditions, such as hand priming, hand grasping, and hand-action centrality. Finally, whole-brain representational similarity analysis revealed that the similarity of multivoxel object response patterns in left lateral occipitotemporal cortex selectively predicted the degree to which objects were rated as being controlled by and extending the body. Together, these results reveal a clustering of body and object effector representations, indicating that the organization of object representations in high-level visual cortex partly reflects how objects relate to the body.

Figure 1.

Conditions tested in Experiments 1–4. Examples of object categories tested in the four experiments. Different color codes represent stimuli tested in each experiment (yellow for Experiment 1, blue for Experiment 2, magenta for Experiment 3, and green for Experiment 4) and across more than one experiment (orange for Experiments 1, 3, and 4; dark magenta for Experiments 1–4).

Figure 2.

Results of ROI analysis. A, Individual-participant LOTC-hand ([hands > whole bodies + chairs], and LOTC-body ([whole bodies > hands + chairs]) ROIs are shown for five representative participants. ROIs were restricted to cubes of 10 mm width centered on the activation peak (threshold, p < 0.001, uncorrected). LH, Left hemisphere. B, Mean responses for all stimulus categories in all experiments are rank ordered by response amplitude in LOTC-hand and LOTC-body. Bars for each condition are color coded according to the following groups: bodies include noneffector human body parts (torsos, bodies); body effectors include body parts used as effectors (active hand postures, resting hands, feet, upper limbs, lower limbs); object effectors include body-extending object effectors (mechanical tools, act-with objects, sport-related act-with objects); and objects include all noneffector objects (act-on objects, small/portable musical instruments, big/nonportable musical instruments, small/portable objects, large/nonportable objects, high-graspability objects, low-graspability objects, sport-related objects, chairs, vehicles). Faces and animals were excluded from this classification and are shown in white. As a measure of selectivity, all conditions were tested within experiments against the control category chairs (chairs were included in all experiments) using two-tailed pairwise t tests (e.g., hands in Experiment 1 vs chairs in Experiment 1). Object categories that survived the Bonferroni-corrected threshold of p < 0.0016 (p = 0.05/36 stimulus conditions) are highlighted with gray background. In LOTC-body, hands (Experiment 2) also survived Bonferroni correction, and the corrected p value is reported above the bar. Numbers below each bar represent the experiment number (1–4) of each condition. Error bars indicate SEM.

Figure 3.

Results of regression analysis. A, Single regression analyses were used to assess how variability in the fMRI responses in LOTC-hand and LOTC-body relate to the variability in the four rated action-related object dimensions (body extension, hand movement prime, hand grasping, and hand-action centrality) rated by an independent group of participants (Group 1, n = 15). Results revealed that the response profiles of LOTC-hand and LOTC-body were most strongly related to the body extension dimension (blue color-coded bar graph). Error bars indicate SEM. B, The mean fMRI responses (dependent variables) to the conditions included in the regression analyses are rank ordered by response amplitude in LOTC-hand and LOTC-body. For conditions that were presented in multiple experiments (tools and chairs), responses were averaged across these experiments. Error bars indicate SEM.

Figure 4.

Results of multivoxel pattern analysis. Multivoxel correlation matrices were computed in a left LOTC region, functionally defined in each individual participant by contrasting the average response to hands and bodies relative to chairs (see Materials and Methods). In each experiment, activity patterns for the hand and body conditions were correlated with activity patterns for the other object conditions using a split-half (odd/even) comparison. The 2 × 2 matrix (left side) shows within- and between-category correlations for hands and bodies. The 2 × 6 matrix (right side) shows correlations for hands and bodies (rows) with the other objects categories (columns) for each experiment. Each cell of the matrix represents the mean correlation value across participants within each experiment. Warm colors represent positive correlations, and cold colors represent negative correlations.

Figure 5.

Results of univariate whole-brain analysis. Univariate group average activation maps (combining all four experiments) for the contrasts of body effectors versus bodies (dark blue color-coded) and object effectors versus objects (light blue color-coded) are shown at the Bonferroni-corrected threshold of p < 0.05.

Figure 6.

Results of representational similarity analysis. A, Mean representational dissimilarity matrices for the four action-related behavioral dimensions (body extension, hand movement prime, hand grasping, and hand-action centrality) computed as the absolute distance between the ratings of each object pair. Only pairs available in the neural dissimilarity matrix were included in the analysis (see Materials and Methods). B, Mean neural dissimilarity matrices (1-correlation) for brain regions (left LOTC, left IPS, right IPS, left PM; C) in which the neural dissimilarity matrix correlated more with the body extension dissimilarity matrix than with any of the other three behavioral dissimilarity matrices (hand movement prime, hand grasping, and hand-action centrality). These matrices are included for illustration purposes only. C, Results of whole-brain RSA, showing the conjunction contrast between the body extension dimension and each of the other three dimensions, revealed a cluster in left LOTC (at p < 0.05, Bonferroni corrected). Multivoxel response patterns in this LOTC region were thus relatively similar between objects that were rated as relatively similar on the body extension dimension.