Object ensemble processing in human anterior-medial ventral visual cortex

Abstract

Our visual system can extract summary statistics from large collections of similar objects without forming detailed representations of the individual objects in the ensemble. Such object ensemble representation is adaptive and allows us to overcome the capacity limitation associated with representing specific objects. Surprisingly, little is known about the neural mechanisms supporting such object ensemble representation. Here we showed human observers identical photographs of the same object ensemble, different photographs depicting the same ensemble, or different photographs depicting different ensembles. We observed fMRI adaptation in anterior-medial ventral visual cortex whenever object ensemble statistics repeated, even when local image features differed across photographs. Interestingly, such object ensemble processing is closely related to texture and scene processing in the brain. In contrast, the lateral occipital area, a region involved in object-shape processing, showed adaptation only when identical photographs were repeated. These results provide the first step toward understanding the neural underpinnings of real-world object ensemble representation.

Figure 1.

Example stimuli and results (N = 12) from experiment 1. a, Example stimuli used in the experiment. Based on common occurrence and ease of availability, 20 different living object ensemble images and 20 different nonliving texture images were used. In each trial, observers saw a sequential presentation of three images that were all identical (gray boxes), all different (red boxes), or shared object ensemble or surface texture statistics (blue boxes). To ensure attention to the images, observers were required to press a button on the disappearance of the third image in the sequence. b, Results from experiment 1. fMRI responses were extracted from independently localized object- (LO) and scene-sensitive (PPA) areas of cortex. PPA exhibited similar response patterns to both object ensembles and surface textures and showed equivalent levels of adaptation (i.e., a reduction in activation compared with the different condition) in the identical and the shared conditions when either object ensemble or surface texture statistics were repeated. In contrast, LO response patterns differed between object ensembles and surface textures, exhibiting an equivalent release from adaptation in the shared and different object ensemble conditions, where changes to local shape information are evident, but exhibited insensitivity to changes in surface textures due to a lack of closed contours in those images. Error bars represent within-subject SEs. ns, Not significant. c, Additional examples of stimuli used in experiment 1. *p < 0.05.

Figure 2.

Examples of ROIs in individual observers. The scene-selective PPA (Talairach coordinates for the specific ROI examples shown x, y, z for right/left; +24/−24, −40/−41, −3/−6), RSC (+15/−16, −53/−56, +14/+9), and TOS (+31/−38, −79/−77, +23/+22) were defined by contrasting the activation for scenes against the activation for both faces and objects. The object-selective LO (+31/−32, −77/−84, −3/−7) and pFs (+32/−29, −63/−61, −13/−16) were defined by contrasting the activation for objects against the activation for scrambled objects. R, Right; L, left.

Figure 3.

Example stimuli and results (N = 14) from experiment 2. a, Example stimuli used in the experiment. A new set of images, different from that used in experiment 1, was used in this experiment. This new set contained 20 object ensemble and 20 texture images, each containing 10 living and 10 nonliving examples. All stimuli were presented in grayscale to remove the repetition of colors when ensemble statistics were repeated. In each trial, observers saw a sequence of two images that were identical (gray boxes), different (red boxes), or shared object ensemble or surface texture statistics (blue boxes). Observers were asked to categorize each trial as identical, shared, or different. b, Results from experiment 2. Replicating results from experiment 1, PPA again showed equivalent levels of adaptation when object ensemble or texture features were repeated, and LO showed adaptation only when the local shape/contours were identical in the object ensemble images but showed no sensitivity to surface texture manipulations. Error bars represent within-subject SEs. ns, Not significant. c, Additional examples of stimuli used in experiment 2. *p < 0.05; **p < 0.01.

Figure 4.

Stimuli and results (N = 14) from experiment 3. a, Example stimuli used in the experiment. The same stimuli, conditions, and tasks from experiment 2 were used here except the images were shown in full color, and, to investigate how image size changes affect brain responses, the second image in each trial was approximately two-thirds the size of the first image. b, Results from experiment 3. Replicating results from experiments 1 and 2, PPA again showed equivalent levels of adaptation when object ensemble or surface texture statistics were repeated, and LO showed adaptation only when the local shape/contours were identical in the object ensemble images but showed no sensitivity to surface texture manipulations. Error bars represent within-subject SEs. ns, Not significant. c, Additional examples of stimuli used in experiment 2. *p < 0.05; **p < 0.01.

Figure 5.

PPA and LO results for the four conditions in the ensemble/texture localizer. In PPA, the two intact image conditions did not differ from each other, nor did the two scrambled image conditions differ from each other. Although spatial frequency and other low-level image information (such as contrast and luminance) were equated between intact and scrambled images, intact images elicited significantly higher responses than scrambled images. For comparison, activations for scenes and single objects from the object/scene localizer are also plotted (computed in individual observers by defining PPA using the first run of the object/scene localizer and then extracting independent data from this region using the last run of the object/scene localizer). Scenes elicit the highest activation in PPA compared with objects, ensembles, and textures. Thus, although we show in this study that PPA is the key brain region mediating the representation of ensemble and texture statistics, scenes still seem to be the most effective stimuli in driving PPA response. For completeness, responses in LO, computed using the same method described for PPA, were included as well. ***p < 0.001.

Figure 6.

Stimuli and results (N = 11) from experiment 4. a, Example stimuli used in the experiment. Only object ensembles made of black wooden beads were used here. In each trial, observers saw two images that were identical (gray boxes), shared object ensemble features (i.e., different photographs of the same type of beads; blue boxes), or different (i.e., photographs of beads that differed in the shape of the individual ensemble elements; red boxes). The surface texture and the material properties of the individual objects in the ensembles were thus identical in all conditions. The same image categorization task used in experiments 2 and 3 was used here. b, Results from experiment 4. Despite texture/material repetition of the ensembles across the three conditions, PPA again showed adaptation when ensemble statistics were repeated but a release from adaptation when the shape of the beads changed between ensembles. Unlike the previous three experiments, LO did not show any sensitivity to our manipulations, possibly because half of the beads used in the experiment were approximately circular, resulting in minimal contour changes between different images depicting either the same or different ensembles. Error bars represent within-subject SEs. ns, Not significant. c, Additional stimuli used in experiment 4. *p < 0.05.

Figure 7.

a, Group overlap of the object ensemble/surface texture regions and PPA and LO. To localize visual areas naturally activated when object ensemble and surface textures are processed, brain responses for viewing object ensembles and surface textures were contrasted with those for viewing phase-scrambled versions of these same images. Two main regions of activation were located (shown in purple), with one located laterally and the other located ventrally originating from the parahippocampal gyrus and extending posteriorly along the collateral sulcus. Although the anterior-medial ventral region (Talairach coordinates, x, y, z for right/left: +21/−23, − 50/−52, −6/−9) overlapped greatly with PPA (yellow; + 21/−21, −39/−40, −7/−7, defined by contrasting scenes with faces and everyday objects), the lateral region (+33/−37, − 77/−74, −1/−6) overlapped greatly with LO (green; 36/−36, −75/−75, −3/−4, defined by contrasting everyday objects with phase-scrambled versions of these same images). The large overlap between the different brain regions justifies our selection of PPA and LO as the main ROIs in investigating the neural underpinnings of object ensemble processing. All regions are displayed at p < 0.001, uncorrected. b, Regions differentially activated for object ensembles or surface textures. The only regions that were more active (group data, displayed at p < 0.001, uncorrected) in the visual occipitotemporal cortex for object ensembles than for surface textures were located in LO (+33/−35, −80/−78, −1/−4) and early visual cortex (8, −86, 0). No regions were more active for textures than for ensembles. c, Common region of overlap between PPA (defined using the scenes vs faces and objects contrast) and the ventral ensemble/texture region (defined using the intact vs scrambled ensembles and textures contrast), both at the group level and displayed at p < 0.001 (+21/−22, −41/−44, −7/−6) and p < 0.01 (+ 20/−21, −44/−41, −6/−6), both uncorrected. As the statistical threshold is relaxed, the common region of activation extends more posteriorly. R, Right.

Figure 8.

Top, Common regions of overlap for PPA (defined using the object/scene localizer) and the ventral ensemble/texture region (defined using the ensemble/texture localizer), shown in five representative observers. Bottom, Common regions of overlap for LO (defined using the object/scene localizer) and the lateral ensemble/texture region (defined using the ensemble/texture localizer), shown in the same five observers. All regions are displayed at p < 0.001, uncorrected. Talairach coordinates are given under each brain. R, Right; S, subject.

Figure 9.

Regions that exhibit adaptation for repetitions of object ensemble and surface texture statistics (i.e., lower response for the identical and the shared conditions than for the different condition), plotted separately for each of the four adaptation experiments and shown as outlines to illustrate the overlap across experiments. The locations of the regions are consistent across the four experiments and reside in the anterior-medial part of ventral visual cortex, extending along the collateral sulcus and the parahippocampal gyrus (Talairach coordinates for experiment 1, anterior collateral sulcus/parahippocampal gyrus [aCoS/PG], x, y, z for right/left: +24/−25, −49/−56, −5/−11; right posterior collateral sulcus [pCoS]: 21, −76, −12; for experiment 2, aCoS/PG: +21/−26, −37/−46, −15/−8; right pCoS: +21, −69, −12; for experiment 3, aCoS/PG: +22/−20, −41/−49, −14/−12; right pCoS: 24, −66, −13; for experiment 4, aCoS/PG: +28/−30, −40/−53, −16/−11; right pCoS: 22, −63, −9). These results provide additional support for our choice of PPA as the main ROI for examining object ensemble and surface texture processing in the visual cortex. R, Right.

Figure 10.

Adaptation results in RSC, TOS, and pFs, shown for all four adaptation experiments. Responses to object ensembles and textures in RSC and TOS were not as consistent as those observed in PPA, suggesting that RSC and TOS are unlikely to play significant roles in ensemble and texture processing. This suggests that PPA is involved in both spatial and nonspatial aspects of visual processing, but RSC and TOS may only participate in spatial aspects of visual processing. Depending on the experiment and the stimulus condition, pFs responses were either similar to LO (ensembles in experiments 1 and 2) or PPA (ensembles in experiment 1, and basic adaptation effect for textures in experiment 1). This suggests that pFs, which is anterior to LO but posterior to PPA, may be a “transition zone” whose function is transitioning from processing shapes to processing the statistical information contained in ensembles and textures. Exp, Experiment; ns, not significant. Error bars represent within-subject SEs. *p < 0.05; **p < 0.01.