How (and why) the visual control of action differs from visual perception

Abstract

Vision not only provides us with detailed knowledge of the world beyond our bodies, but it also guides our actions with respect to objects and events in that world. The computations required for vision-for-perception are quite different from those required for vision-for-action. The former uses relational metrics and scene-based frames of reference while the latter uses absolute metrics and effector-based frames of reference. These competing demands on vision have shaped the organization of the visual pathways in the primate brain, particularly within the visual areas of the cerebral cortex. The ventral 'perceptual' stream, projecting from early visual areas to inferior temporal cortex, helps to construct the rich and detailed visual representations of the world that allow us to identify objects and events, attach meaning and significance to them and establish their causal relations. By contrast, the dorsal 'action' stream, projecting from early visual areas to the posterior parietal cortex, plays a critical role in the real-time control of action, transforming information about the location and disposition of goal objects into the coordinate frames of the effectors being used to perform the action. The idea of two visual systems in a single brain might seem initially counterintuitive. Our visual experience of the world is so compelling that it is hard to believe that some other quite independent visual signal-one that we are unaware of-is guiding our movements. But evidence from a broad range of studies from neuropsychology to neuroimaging has shown that the visual signals that give us our experience of objects and events in the world are not the same ones that control our actions.

Keywords: dorsal stream; ventral stream; visual illusions; visual perception; visuomotor control.

Figure 1.

Neuroimaging of visually guided grasping. The two panels in the upper left show the ‘grasparatus’ that James et al. [17] used to present graspable objects to participants in the scanner. Different rear-illuminated target objects could be presented from trial to trial on the grasparatus by rotating the pneumatically driven eight-sided drum. Participants were instructed either to grasp the object or to simply reach out and touch it with the back of the hand. Note that the experiment would normally be run in the dark with only the illuminated target object visible. The graph at the bottom left side shows the time course of activations in hAIP when a healthy participant reached out and grasped an object (green) versus when she reached out and simply touched the object with the back of her hand without forming a grasp (red). The location of hAIP is mapped onto the left hemisphere of the brain (and inset) on the right side of the figure. (The other grasp-specific activations shown on the brain are in the motor and somatosensory cortices.) Note that in area LO (surrounded by the blue rectangle), there is no differential activation for grasping. Adapted from Culham et al. [26].

Figure 2.

The effect of a size-contrast illusion on perception and action. Panel (a) shows the traditional Ebbinghaus illusion in which the central circle in the annulus of larger circles is typically seen as smaller than the central circle in the annulus of smaller circles, even though both central circles are actually the same size. Panel (b) shows the same display, except that the central circle in the annulus of larger circles has been made slightly larger. As a consequence, the two central circles now appear to be the same size. Panel (c) shows the experimental set-up in which the participant was required to reach out and pick up one of two discs that were placed on the illusory display. Panel (d) shows for grip aperture for two trials in which the participant picked up the small disc on one trial and the large disc on another from the display shown in panel (b). Even though the two central discs were perceived as being the same size, the grip aperture in flight reflected the real not the apparent size of the discs. Adapted from Aglioti et al. [43].

Figure 3.

The Ponzo illusion experiment by Ganel et al. [47]. The top diagram shows the experimental display. Although the rectangular target object located on the right-hand side of the display is perceived as being longer than the one on the left, it is actually shorter. The graph below shows the mean maximum grip aperture and mean perceptual estimations of the participants. Performance when the targets were placed on the Ponzo display is shown on the left and performance when the targets were placed on a control grid with no illusion is shown on the right. Grip aperture was unaffected by the Ponzo illusion and was tuned to the actual length of the target objects. Perceptual estimations, however, were affected by the Ponzo illusion and reflected the perceived not the actual length of the target objects. Adapted from Ganel et al. [47].