Seeing without a Scene: Neurological Observations on the Origin and Function of the Dorsal Visual Stream

Abstract

In all vertebrates, visual signals from each visual field project to the opposite midbrain tectum (called the superior colliculus in mammals). The tectum/colliculus computes visual salience to select targets for context-contingent visually guided behavior: a frog will orient toward a small, moving stimulus (insect prey) but away from a large, looming stimulus (a predator). In mammals, visual signals competing for behavioral salience are also transmitted to the visual cortex, where they are integrated with collicular signals and then projected via the dorsal visual stream to the parietal and frontal cortices. To control visually guided behavior, visual signals must be encoded in body-centered (egocentric) coordinates, and so visual signals must be integrated with information encoding eye position in the orbit-where the individual is looking. Eye position information is derived from copies of eye movement signals transmitted from the colliculus to the frontal and parietal cortices. In the intraparietal cortex of the dorsal stream, eye movement signals from the colliculus are used to predict the sensory consequences of action. These eye position signals are integrated with retinotopic visual signals to generate scaffolding for a visual scene that contains goal-relevant objects that are seen to have spatial relationships with each other and with the observer. Patients with degeneration of the superior colliculus, although they can see, behave as though they are blind. Bilateral damage to the intraparietal cortex of the dorsal stream causes the visual scene to disappear, leaving awareness of only one object that is lost in space. This tutorial considers what we have learned from patients with damage to the colliculus, or to the intraparietal cortex, about how the phylogenetically older midbrain and the newer mammalian dorsal cortical visual stream jointly coordinate the experience of a spatially and temporally coherent visual scene.

Keywords: Bálint’s syndrome; hemispatial neglect; parietal lobe; progressive supranuclear palsy; superior colliculus.

Figure 1

Drawings of the lesions of the patient described by Bálint (1909). Adapted from Rafal (2001, Fig. 3, p. 124). Arrows show the intraparietal sulcus.

Figure 2

Things appear and disappear. (A) This woman, who had recently experienced bilateral parietal strokes, initially failed to notice a pipe that I held to the right of her fixation. (B) When I attracted her attention by moving it, she looked it, and she was able to follow it when it was moved to her left (C). When the pipe was moved again (D), she did not follow it with her eyes and reported seeing the pipe; however (E), when the pipe was removed and replaced by a pen held eccentric to her fixation, she failed to look at it, and (F) when asked to identify it, reported that “it’s still a pipe”. (G) When it was moved into her fixation in the next frame, she correctly identified it as a pen, but when it was then moved again, it disappeared (H,I) and she could not find it. Adapted from Rafal (2001, Fig. 9, p. 132).

Figure 3

Simultaneous agnosia in a patient with Bálint’s syndrome, showing attentional capture by a distant stimulus. From Rafal (2001, Fig. 5, p. 126). From a video by Perpetua Productions. This patient could only see one object at a time. He could see a comb or a spoon, but not both, even though they were projected on the same region of his retinae. At one point both the comb and the spoon disappeared when his eyes locked on some fine chalk marks on a blackboard behind the examiner.

Figure 4

Optic ataxia (misreaching) in a patient with Bálint’s syndrome. Although this patient said that he saw a spoon and was looking at it straight in front of him, he was unable to accurately reach toward it or to orient his hand appropriately to grasp it. From a video by Perpetua Productions.

Figure 5

A virtual dissection, using probabilistic diffusion imaging tractography, showing connections between the lateral geniculate nucleus and the superior colliculus (red) and projections from the primary visual cortex to the superior colliculus via the brachium of the superior colliculus (green).

Figure 6

Virtual dissection, with probabalistic diffusion tractography, showing that the dorsal stream is embedded in a network with the superior colliculus, on the back of the midbrain (yellow), as its hub. The dorsal stream refers to projections (not shown) from the primary visual cortex (V1) to the dorsal extrastriate cortex including the intraparietal cortex (IPCx) and the frontal eye field (FEF). The colliculus receives visual signals from the retina both directly, via the retinotectal tract (pink), and from the primary visual cortex (light blue), with both visual pathways entering the colliculus via the brachium of the superior colliculus. In addition to receiving visual signals from the magnocellular and parvocellular layers of the lateral geniculate nucleus, the visual cortex also receives visual signals from the superficial layers of the superior colliculus via the koniocellular layers of the lateral geniculate nucleus (not shown). The intraparietal cortex also receives information about the position of the eyes in the orbit: When the superior colliculus sends eye movement commands to eye movement generators in the brain stem, a copy of the command is sent, via recurrent axon collaterals, to the intraparietal cortex via the frontal eye fields (red). The region of the frontal eye fields that receives eye movement signals from the colliculus projects to the intraparietal cortex as a component of the superior longitudinal fasciculus (dark blue); and the intraparietal cortex, in turn, is connected to the superior colliculus (green). The frontal eye field also sends endogenously generated commands for voluntary eye movements to the superior colliculus—which is the final common pathway for initiating orienting responses.

Figure 7

Three-dimensional reconstructions of the retinotectal tract and projections from the superior colliculus to the amygdala. (Left): Viewed from the left front, the retinotectal tract (red) in the human brain was virtually dissected using probabilistic diffusion imaging tractography. A seed mask was placed in the optic tract (OT) and a target mask was placed in the superior colliculus (SC). (Right): The retinotectal tract in blue, viewed from above, and projections (red) from the superior colliculus to the amygdala (Amg).

Figure 8

Newborn infants will track the moving face-like figure on the left but not the inverted figure on the right. Also, a baby will not track the figure on the left if the contrast is inverted (white dots forming a triangle on a black background).

Figure 9

The performance of 3- and 7-month-old infants on a double saccade task (Gilmore and Johnson 1997). (Left): The temporal sequence of a trial in the double saccade task, shown from top to bottom. Participants fixate the + at the top of the display. Simultaneous with the offset of the +, two targets are presented in sequence, with the first of the two targets appearing randomly on the left or right. Participants are instructed to look at the locations of the two targets in the order they appear. In this illustration, a target on the left is followed by a target on the right. Critically, both targets are flashed only briefly in quick succession such that both targets disappear before the first eye movement can be initiated. (Right): The performance of 3-month-old (top) and 7-month-old (bottom) infants on the double-step saccade task. The 3-month-old infants made eye movements toward the location that the target had appeared on their retinae. The 7-month-old infants’ performance reflects the effect of an extra-retinal signal (the effect of eye movement) such that the second saccade is made to the actual location in the scene where the second target had appeared.

Figure 10

(Top) A virtual dissection, using probabilistic diffusion imaging tractography, of connections (red) between the superior colliculus and the frontal eye fields (FEF). The FEF is located on the posterior part of the middle frontal gyrus at the intersection of the superior frontal sulcus (SFS) and the precentral sulcus. (Bottom) The figure on the left shows the termination (red), in the FEF, of projections from neurons in the mediodorsal thalamus that transmit copies of eye movement signals from the deep layers of the superior colliculus. The bottom of Figure 10 shows that the termination point (red) of this pathway from the colliculus to the frontal eye field in the human brain (bottom left) corresponds closely with that demonstrated electrophysiologically by Sommer and Wurtz (2004) in two monkeys (bottom right). When comparing sulcal landmarks in the two species, note that in the monkey brain these projections synapse on FEF neurons located midway between the end of the principal sulcus and the curve of the arcuate sulcus (gray ovals). In the human brain, the homolog of the arcuate sulcus is formed by the intersecting superior frontal and precentral sulci. The sulcus in the human brain, labeled “P” in the bottom left figure, may be a remnant homolog of the monkey principal sulcus (P).

Figure 11

Series from a video of the same patient with Bálint’s syndrome shown in Figure 2. (Top): I first asked the patient if I was wearing glasses, and she replied, “I think you are”. (Middle): I then ducked away, took off my glasses, and asked her again if I was wearing glasses (Middle), and she replied that she thought I was wearing glasses. (Bottom): After ducking away again and putting on my glasses, she had “only a guess”. Although my glasses and my face were plotted at the same location on her retinae, she could see my glasses, or she could see my face, but she could not see both (Adapted from Rafal 2001, Fig. 2, p. 123).

Figure 12

Hemispatial neglect in a patient with hemispatial neglect (See text) (adapted from a video by Perpetua Productions).

Figure 13

Object-based visual extinction. (Top): Both fingers wiggle. The patient looks to the finger wiggling on his right, says “your left hand”, and does not detect the finger wiggling on his left. (Middle): When the examiner rotates counterclockwise and wiggles fingers on both hands, the patient looks upward, and he says, “your left hand”. (Bottom): When the examiner rotates clockwise and wiggles fingers on both hands, the patient looks down, and he says, “your left hand”. There is neglect of the left side of the object that the patient is viewing (the examiner) regardless of where the left side of the object is in retinotopic coordinates (adapted from a video by Perpetua Productions).

Figure 14

Extinction and stimulus repetition. Patients with visual extinction are more likely to demonstrate visual extinction if the two competing items are the same than if they are different; see text (from a video by Perpetua Productions).

Figure 15

The patient was asked to copy the figure shown at the top of each panel. The patient’s copies are shown on the bottom of each panel before (left) and after (right) prism adaptation.

Figure 16

Representational neglect reduced after treatment with prism adaptation (adapted from Rode et al. 2001, Fig. 2, p. 1253).

Figure 17

Duplex seeing: a double dissociation (see text). (Left): Adapted from Halligan et al. (1992, Fig. 1, p. 127). (Right): Adapted from Ingle (2004b, Fig. 7, p. 38).