Spatial updating and the maintenance of visual constancy

Abstract

Spatial updating is the means by which we keep track of the locations of objects in space even as we move. Four decades of research have shown that humans and non-human primates can take the amplitude and direction of intervening movements into account, including saccades (both head-fixed and head-free), pursuit, whole-body rotations and translations. At the neuronal level, spatial updating is thought to be maintained by receptive field locations that shift with changes in gaze, and evidence for such shifts has been shown in several cortical areas. These regions receive information about the intervening movement from several sources including motor efference copies when a voluntary movement is made and vestibular/somatosensory signals when the body is in motion. Many of these updating signals arise from brainstem regions that monitor our ongoing movements and subsequently transmit this information to the cortex via pathways that likely include the thalamus. Several issues of debate include (1) the relative contribution of extra-retinal sensory and efference copy signals to spatial updating, (2) the source of an updating signal for real life, three-dimensional motion that cannot arise from brain areas encoding only two-dimensional commands, and (3) the reference frames used by the brain to integrate updating signals from various sources. This review highlights the relevant spatial updating studies and provides a summary of the field today. We find that spatial constancy is maintained by a highly evolved neural mechanism that keeps track of our movements, transmits this information to relevant brain regions, and then uses this information to change the way in which single neurons respond. In this way, we are able to keep track of relevant objects in the outside world and interact with them in meaningful ways.

Figure 1

Updating examples. A. Non-spatial example of updating a checkbook. An initial amount of €100 is updated to a value of €500 after a deposit of €400. B. Spatial example of updating the location of a visual target. A target initially seen 15° to the right, is updated to a location 25° to the right after an intervening eye movement of 10° to the left.

Figure 2

Spatial updating in a classic double-step saccade paradigm. When fixating a target (●), the retinal errors (RE1, RE2) caused by two other targets (T1, T2) are the same in both the no updating (A) and updating (B) conditions. A. Without updating, the movement to the first target (ME1) is made based on RE1, and the movement to the second target (red ME2) is made based on RE2. This leads to a mislocalization of T2. B. With proper updating, the movement to the first target (ME1) is based on RE1, but the movement to the second target (green ME2) is based both on RE2 and ME1. Taking the amplitude and direction of the first movement (ME1) into account leads to the correct localization of T2.

Figure 3

Neural evidence for spatial updating in LIP. A. A cell in LIP responds when a visual target falls within its receptive field. Data aligned on visual stimulus onset. B. A cell in LIP responds when a saccadic eye movement brings the cell’s receptive field onto an illuminated target. Data aligned on saccade onset. C. Some cells begin to respond to an impending shift in the cell’s receptive field even before the eye movement begins. Bottom-right raster plot and histogram are aligned on the onset of the saccade. D. Some cells respond when their receptive field shifts to a location in which a visual target was previously illuminated. Open flash symbol indicates that the stimulus was extinguished before the eyes moved. Data aligned on saccade onset. Replotted with permission from Duhamel et al., 1992a.

Figure 4

Updating of saccades to targets presented during pursuit. The first saccade to a visual target flashed during pursuit is inaccurate if generated with short latency, but becomes increasingly accurate with longer duration latencies. Second, third and fourth saccades are generally accurate. A compensation index of 1 indicates perfect updating. Modified with permission from Blohm et al., 1xxx.

Figure 5

Updating for roll rotations. A. A subject in a static roll position is shown a briefly flashed target (●), and is then rotated to an upright orientations. If the motor error to the location of the remembered target is based only on the retinal error caused by the target (red ME = RE), then the target is mislocalized. If the motor error is based on both the retinal error (RE) and the amplitude and direction of the intervening roll rotation (i.e., 45° clockwise), then the target is correctly localized (green ME). B. When subjects are rolled about the naso-occipital axis from an upright orientation, saccades to remembered target locations are accurate, independent of the body tilt. The average slope across subjects was 0.07. Best fit slopes for each subject are indicated by a dashed line. A slope of 0 indicates perfect updating, while a slope of 1 (solid line) indicates no updating. Replotted with permission from Klier et al., 1995. C. When subjects are rolled about the naso-occipital axis in a supine orientation, saccades are much less accurate. The average slope across subjects was 0.70. Dashed lines and solid lines as in B. Replotted with permission from Klier et al., 1995.

Figure 6

Updating for passive, body-fixed, yaw rotations at various pitch angles. A. As the pitch angle increases (from left to right), the gravity cue available for updating increases. In the upright orientation, the gravity vector remains constant during yaw rotation, while in the supine condition, the gravity vector changes maximally during yaw rotation. Solid lines on head indicate axis of rotation. B. Updating ability was equally good, for all subjects (different symbols), in both the upright and supine orientations. A value of 1 indicates perfect updating, while a value of 0 indicates no updating. Modified and replotted with permission from Klier et al., 2006.

Figure 7

Updating before and after labyrinthine lesions. Updating ability is evaluated, for two animals, by an updating index (1 indicates perfect updating; 0 indicates no updating) before (time = 0) and up to 16 weeks after lesions of the labyrinths. A. Updating performance for yaw rotations. B. Updating for lateral translations. C. Updating for fore-aft translations. Replotted with permission from Wei et al., 2006.

Figure 8

The SC-MD-FEF pathway for updating signals. A. Recording studies delineate a pathway for updating signals from the SC to the FEF via the MD thalamus. Of the three signal types received by the FEF through this pathway, the visual burst arrives too late to be utilized for spatial updating, the delay period activity is too small to account for spatial updating, but the saccadic burst activity is appropriate for spatial updating. Replotted with permission from Sommer and Wurtz, 2004a. B. Inactivation of area MD leads to horizontal mislocalization of the second target in a double-step saccade task. Replotted with permission from Sommer and Wurtz, 2002.

Figure 9

Updating after split-brain experiments. A. Two versions of the double-step saccade task. In both versions, a monkey fixates on FP and two targets (T1, T2) are briefly flashed in the right visual hemifield. Thus both targets are represented in the left hemisphere. Green square: An eye movement to T1, causes the representation of T2 to remain in left hemisphere. Red square: An eye movement to T1, causes T2 to shift into the right hemisphere. B. After the forebrain commissure is severed, monkeys have difficulty localizing T2 when its representation crosses form one hemisphere to the other (red stippling). C. After some time, the monkeys recover their cross-hemisphere updating ability (red stippling). Replotted with permission from Berman et al., 2005.

Figure 10

Eye-centered reference frame revealed by fMRI. A subject fixates the origin while two targets are briefly flashed (before 1^st saccade column). The final goal (green target) is flashed first, followed by the first fixation point (red target). The subject first makes a saccade to the red target (after 1^st saccade column) and subsequently makes a saccade to the green target (not shown). In some conditions, the representation of the final goal (green target) is kept in the same hemisphere (e.g., goal stays in right hemisphere in the green RR condition and goal stays in the left hemisphere in the red LL condition). In other conditions, the saccade to the first fixation point (red target) causes the final goal (green target) to switch its location from one hemisphere to the other (e.g., from right to left in black RL condition and from left to right in blue LR condition). B. Activity that stays in the right hemisphere is shown by the green trace (RR condition), while activity in the left hemisphere is shown by the red trace (LL condition). Activity can be seen jumping from one hemisphere to the other with the black (RL) and blue (LR) conditions. For example, in the left PPC, the black trace follows the green trace during 1st delay period, but follow the red trace during the 2^nd delay period. Replotted with permission from Medendorp et al., 2003a.

Figure 11

Allocentric reference frame depends on gravity cues. A. Subjects show errors in perceiving the orientations of vertical lines in space depending on their tilted roll angle. With small body tilts, subjects misperceive the orientations of vertical lines in a direction opposite to that of their body tilt (E-effect). With larger body tilts, subjects misperceive the orientations of vertical lines in the same direction as their body tilt (A-effect). Replotted with permission from Van Pelt et al., 2005. B. Subjects were rotated by a fixed roll angle (e.g., 120°) from different initial orientations (e.g., option 1 = 60° counterclockwise; option 2 = 0°) to different final orientations (e.g., option 1 = 60° clockwise; option 2 = 120° clockwise). C. Updating ability more closely follows the predictions of an allocentric model (dashed red unity slope) that was based on the perceptual errors in the different final orientations, than the predictions of an egocentric model that is independent of these perceptual errors. Replotted with permission from Van Pelt et al., 2005.