Shape representations in the primate dorsal visual stream

Abstract

The primate visual system extracts object shape information for object recognition in the ventral visual stream. Recent research has demonstrated that object shape is also processed in the dorsal visual stream, which is specialized for spatial vision and the planning of actions. A number of studies have investigated the coding of 2D shape in the anterior intraparietal area (AIP), one of the end-stage areas of the dorsal stream which has been implicated in the extraction of affordances for the purpose of grasping. These findings challenge the current understanding of area AIP as a critical stage in the dorsal stream for the extraction of object affordances. The representation of three-dimensional (3D) shape has been studied in two interconnected areas known to be critical for object grasping: area AIP and area F5a in the ventral premotor cortex (PMv), to which AIP projects. In both areas neurons respond selectively to 3D shape defined by binocular disparity, but the latency of the neural selectivity is approximately 10 ms longer in F5a compared to AIP, consistent with its higher position in the hierarchy of cortical areas. Furthermore, F5a neurons were more sensitive to small amplitudes of 3D curvature and could detect subtle differences in 3D structure more reliably than AIP neurons. In both areas, 3D-shape selective neurons were co-localized with neurons showing motor-related activity during object grasping in the dark, indicating a close convergence of visual and motor information on the same clusters of neurons.

Keywords: depth; dorsal stream; macaque; object; parietal cortex; shape; visual cortex.

Figure 1

Cortical areas processing object shape. (A) Overview of the macaque brain illustrating the locations of the areas involved in processing object shape, and the most important connections between these areas. Unidirectional arrows indicate the presumed flow of visual information along the dorsal stream, the bidirectional arrow between AIP and TEs indicates that the direction of information flow is unclear at present. Note that most connections in extrastriate cortex are bidirectional. CIP, caudal intraparietal area; LIP, lateral intraparietal area; AIP, anterior intraparietal area; F5a, anterior subsector of area F5; TEs, subsector of area TE in the anterior Superior Temporal Sulcus. (B) Schematic flow chart of visual 3D information. Dark boxes indicate areas of the dorsal visual stream, open boxes indicate ventral stream areas, hatched boxes indicate areas selective for higher-order disparity. Boxes with question marks indicate unknown areas.

Figure 2

Coding of shape features in AIP. Example AIP neuron responding to an image of an object and to fragments of image contours. In each box the stimulus is illustrated, the color of each box represents the normalized firing rate of the neuron to that stimulus (maximum response was 29 spikes/s). Top row: intact object contour, second row: the four fragments derived from subdividing the object contour into four fragments along the main axes of the shape (four-fragment stimuli). Third and fourth row: eight- and 16-fragment stimuli. Each contour fragment is connected to the stimulus from which it was derived. The original object images from which the contours were derived are illustrated on the left side (arrows pointing to their respective contour stimuli). All stimuli were presented at the center of the RF. Reproduced with permission from (Romero et al., 2014).

Figure 3

Example random dot stereograms. The monocular images are illustrated below a 3D rendering of two depth stimuli (Gaussian depth profile and Inclined depth profile).

Figure 4

Example neuron recorded in area AIP responding selectively to the depth structure of surfaces. The top row shows the peristimulus-time histograms (PSTHs) of the responses to a convex depth profile, the bottom row the responses to a concave depth profile. Stereo: binocular presentation with disparity. Left eye: monocular presentation to the left eye. Right eye: monocular presentation to the right eye. The horizontal line below the PSTHs indicates the duration of stimulus presentation (800 ms). This neuron preserved its selectivity across positions in depth (data not shown).

Figure 5

Higher-order disparity selective F5a neuron. The PSTHs in the top row illustrate the responses of a single F5a neuron to a convex depth profile at five different positions in depth, the bottom row the responses to a concave depth profile at the same positions in depth. Below the PSTHs is a schematic illustration of the stimulus presentation indicating the positions in depth. Reproduced with permission from (Theys et al., 2012a).

Figure 6

Comparison between AIP and F5a. (A) Average responses of a population of F5a neurons to various approximations of the smoothly curved surfaces (on the left). White bars indicate the responses to the preferred depth profile, filled bars the responses to the non-preferred depth profile. Below the bar graph are schematic illustrations of the various approximations: linear, three different discrete approximations, and first-order (tilted plane). The asterisks indicate significant differences in response. (B) Average responses of a population of AIP neurons to the same stimuli. Same conventions as in (A). Adapted with permission from (Srivastava et al., 2009).