The neural representation of objects formed through the spatiotemporal integration of visual transients

Abstract

Oftentimes, objects are only partially and transiently visible as parts of them become occluded during observer or object motion. The visual system can integrate such object fragments across space and time into perceptual wholes or spatiotemporal objects. This integrative and dynamic process may involve both ventral and dorsal visual processing pathways, along which shape and spatial representations are thought to arise. We measured fMRI BOLD response to spatiotemporal objects and used multi-voxel pattern analysis (MVPA) to decode shape information across 20 topographic regions of visual cortex. Object identity could be decoded throughout visual cortex, including intermediate (V3A, V3B, hV4, LO1-2,) and dorsal (TO1-2, and IPS0-1) visual areas. Shape-specific information, therefore, may not be limited to early and ventral visual areas, particularly when it is dynamic and must be integrated. Contrary to the classic view that the representation of objects is the purview of the ventral stream, intermediate and dorsal areas may play a distinct and critical role in the construction of object representations across space and time.

Keywords: Dorsal stream; MVPA; Shape perception; Spatiotemporal objects; V3A/B.

Figure 1. Stimulus displays exemplifying spatiotemporal boundary formation (SBF) as used in the current experiments

A. An invisible object (red dotted circle, Frame 1) expanded and contracted. Elements entering the boundary of the object (blue arrows, Frame 2) rotated or were displaced by a small amount. The resulting percept (Frame 3) was of expanding and contracting illusory contours. B. The three shapes used in the experiment. The boundaries of the third shape could not be recovered because of the rapid modulation of the contour relative to the density of the background elements. The resulting percept was of flickering elements in a ring-like configuration, but without a clearly-defined form as for squares and circles. This served as the control, no-shape condition.

Figure 2. Regions of interest for a single subject's left hemisphere shown on an inflated cortical surface

Early and ventral visual areas can be seen in the image on the left; intermediate and dorsal areas in the image on the right.

Figure 3. Retinotopy for a single sample subject on an inflated cortical surfaces

The top row depicts intermediate and dorsal regions. The bottom row depicts ventral regions. The left column corresponds to the left hemisphere (LH) and the right column to the right (RH). Note that thresholds vary across the four images to best illustrate the boundaries between ROIs.

Figure 4. Activation across conditions and ROIs

Data are shown for the no-shape (white), circle (light gray), and square (dark gray) conditions for early, intermediate, ventral, and dorsal visual areas. Error bars are standard errors. Normalized (z-scored) % signal change across all voxels in the specified ROI, averaged across hemispheres, element transformations (rotation and displacement), and across all subjects. Stars indicate significant difference in a contrast between shapes (circle and square) and no shape. Diamonds indicate a significant difference in a contrast between circles and squares.

Figure 5

Cross-validated classification accuracy averaged across subjects. Data split by ROI for early, intermediate, ventral, and dorsal visual areas. Data shown for decoding of element rotation vs. displacement (white), shape (circle or square) vs. no shape (light gray), and circle vs. square (dark gray). Chance performance for all classifiers was 50%. Error bars indicate 95% confidence intervals.