Processing multiple visual objects is limited by overlap in neural channels

Abstract

High-level visual categories (e.g., faces, bodies, scenes, and objects) have separable neural representations across the visual cortex. Here, we show that this division of neural resources affects the ability to simultaneously process multiple items. In a behavioral task, we found that performance was superior when items were drawn from different categories (e.g., two faces/two scenes) compared to when items were drawn from one category (e.g., four faces). The magnitude of this mixed-category benefit depended on which stimulus categories were paired together (e.g., faces and scenes showed a greater behavioral benefit than objects and scenes). Using functional neuroimaging (i.e., functional MRI), we showed that the size of the mixed-category benefit was predicted by the amount of separation between neural response patterns, particularly within occipitotemporal cortex. These results suggest that the ability to process multiple items at once is limited by the extent to which those items are represented by separate neural populations.

Keywords: capacity limitations; competition; representational similarity; visual cognition; working memory.

Fig. 1.

(A) Behavioral paradigm. Participants were shown two successively presented displays with four items in each display (Materials and Methods). On the second display, one item was cued (red frame) and participants reported if that item had changed. In the same-category condition, items came from the same stimulus category (e.g., four faces or four scenes). In the mixed-category conditions, items came from two different categories (e.g., two faces and two scenes). (B) Behavioral experiment results. Same-category (light gray) and mixed-category (dark gray) performance is plotted in terms of percent correct for all possible category pairings. Error bars reflect within-subject SEM (48) (*P < 0.05).

Fig. 2.

Visualization of the neural separation procedure. Activation and overlap among the 10% most active voxels for objects and scenes in the occipitotemporal sector is shown in a representative subject. The 10% most active voxels for each category are colored as follows: objects purple, scenes blue. The overlap among these active voxels are shown in yellow. For visualization purposes, this figure shows the most active voxels and overlapping voxels combined across all locations; for the main analysis, overlap was computed separately for each pair of locations.

Fig. 3.

Visualization of our analysis procedure. The center matrix represents the group data from the behavioral experiment with the color of each square corresponding to the size of the mixed-category benefit for that category pairing (Fig. 1). The six remaining matrices correspond to each fMRI participant, with the color of each square corresponding to the amount of neural separation between two categories in occipitotemporal cortex at the 10% activation threshold (Fig. 2). The correlations (r) are shown for each fMRI participant. Note that the r values shown here are the same as those shown in Fig. 4B for occipitotemporal cortex.

Fig. 4.

(A) Visualization of the four sectors from a representative subject. (B) Brain/behavior correlations in every sector for each fMRI participant at the 10% activation threshold, with r values plotted on the y axis. Each bar corresponds to an individual participant. (C) Brain/behavior correlations in every sector for each participant when using the AUC analysis and (D) representational dissimilarity (1 − r).

Fig. 5.

Brain/behavior correlation in each subject for every sector for all possible activation thresholds. Each row shows the results for one of the six individual subjects (in the same order as shown in Fig. 4 for each sector). The x axis corresponds to the percentage of active voxels considered for the neural overlap analysis. The dashed vertical line marks the brain/behavior correlation when considering the 10% most active voxels, corresponding to the data plotted in Fig. 4B.