Using High-Density Electroencephalography to Explore Spatiotemporal Representations of Object Categories in Visual Cortex

Abstract

Visual object perception involves neural processes that unfold over time and recruit multiple regions of the brain. Here, we use high-density EEG to investigate the spatiotemporal representations of object categories across the dorsal and ventral pathways. In , human participants were presented with images from two animate object categories (birds and insects) and two inanimate categories (tools and graspable objects). In , participants viewed images of tools and graspable objects from a different stimulus set, one in which a shape confound that often exists between these categories (elongation) was controlled for. To explore the temporal dynamics of object representations, we employed time-resolved multivariate pattern analysis on the EEG time series data. This was performed at the electrode level as well as in source space of two regions of interest: one encompassing the ventral pathway and another encompassing the dorsal pathway. Our results demonstrate shape, exemplar, and category information can be decoded from the EEG signal. Multivariate pattern analysis within source space revealed that both dorsal and ventral pathways contain information pertaining to shape, inanimate object categories, and animate object categories. Of particular interest, we note striking similarities obtained in both ventral stream and dorsal stream regions of interest. These findings provide insight into the spatio-temporal dynamics of object representation and contribute to a growing literature that has begun to redefine the traditional role of the dorsal pathway.

Figure 1.

The stimuli used in Experiment 1 (A) came from two superordinate categories: animate and inanimate. Each superordinate category was composed of two basic categories (animate: bird and insect; inanimate: tools and graspable objects). Images were processed using the SHINE toolbox to match lower level features, including luminance and spatial frequency. The stimulus set used for Experiment 2 (B) consisted of images from two categories: tools and graspable objects. Of the 10 exemplars that comprised each category, five were “stubby” (i.e., foreshortened) and five were elongated.

Figure 2.

Dorsal (A) and ventral (B) pathway ROIs, shown in the left (L) and right (R) hemispheres on inflated cortical surfaces.

Figure 3.

Decoding performance averaged across participants, plotted for each millisecond of the experimental epoch. Shaded regions around the curve represent SE. The black horizontal line represents chance performance. Asterisks below the line of chance represent time points at which classification was statistically significant after correcting for multiple comparisons. (A) Decoding of individual exemplars. (B) Decoding between the four basic categories (insect vs. bird vs. tool vs. graspable object). (C) Decoding of animacy (animate vs. inanimate). (D) Decoding within animate (bird vs. insect) and within inanimate (tool vs. graspable objects) object categories. Red shading represents time points at which the two curves were significantly different from one another after multiple comparisons correction. Grasp. Obj = graspable object.

Figure 4.

Temporal cross-decoding matrices averaged across participants. Classifiers trained on each time point of the experimental epoch (y axis) were then tested on every other time point (x axis). Values plotted in the matrix represent classifier accuracy at each combination of points. Highlighted regions signify time points that were statistically significant at a false discovery rate corrected p value. (A) Temporal cross-decoding matrix for basic object categories (insect vs. bird vs. tool vs. graspable objects). (B) Temporal cross-decoding matrix for animate (bird vs. insect) object categories. (C) Temporal cross-decoding matrix for animacy (animate vs. inanimate). (D) Temporal cross-decoding matrix for inanimate (tool vs. graspable object) object categories.

Figure 5.

EEG recordings were simulated by projecting dipole activity in dorsal and ventral ROIs (separately) while activity in all other regions was set to zero. Next, source localization was performed on the resulting, simulated waveforms using the same parameters and surfaces as described in the main experiment. The resulting source-localized diploe activity is plotted at different time points in dorsal (teal) and ventral (pink) ROIs. (A) Simulations of ventral activity. (B) Simulations of dorsal activity.

Figure 6.

Decoding performance averaged across participants, plotted across time, and quantified as percent correct. Shaded regions around the curve represent SE. The black horizontal line represents chance performance. Asterisks below the line of chance represent time points at which classification was statistically significant after correcting for multiple comparisons. (A) Decoding of bird versus insects. (B) Decoding of tools versus graspable objects.

Figure 7.

Decoding performance averaged across participants, plotted across time, and quantified as percent correct. Shaded regions around the curve represent SE. The black horizontal line represents chance performance. Red shading highlights time points at which the two curves were significantly different from one another after multiple comparisons correction. Asterisks below the line of chance represent time points at which classification was statistically significant after correcting for multiple comparisons. (A) Decoding of toolness and shape. (B) Decoding tool versus graspable object (long or stubby).

Figure 8.

Temporal cross-decoding matrices averaged across participants. Classifiers trained on each time point of the experimental epoch (y axis) were then tested on every other time point (x axis). Values plotted in the matrix represent classifier accuracy at each combination of points. Highlighted regions signify time points that were statistically significant at a false discovery rate corrected p value. (A) Temporal cross-decoding matrix for toolness (tool vs. graspable object). (B) Temporal cross-decoding matrix for object shape (elongated vs. stubby).

Figure 9.

Classification performance averaged across participants and plotted across time. Shaded regions around the curve represent SE. Regions shaded in red represent time windows during which accuracy differed significantly between the two curves following corrections for multiple comparisons. The black horizontal line represents chance performance. Asterisks below the line of chance represent time points at which classification was statistically significant after correcting for multiple comparisons. (A) Decoding object shape. (B) Decoding toolness without controlling for elongation. (C–D) Decoding toolness while controlling for elongation.