Buildup of spatial information over time and across eye-movements

Abstract

To interact rapidly and effectively with our environment, our brain needs access to a neural representation of the spatial layout of the external world. However, the construction of such a map poses major challenges, as the images on our retinae depend on where the eyes are looking, and shift each time we move our eyes, head and body to explore the world. Research from many laboratories including our own suggests that the visual system does compute spatial maps that are anchored to real-world coordinates. However, the construction of these maps takes time (up to 500ms) and also attentional resources. We discuss research investigating how retinotopic reference frames are transformed into spatiotopic reference-frames, and how this transformation takes time to complete. These results have implications for theories about visual space coordinates and particularly for the current debate about the existence of spatiotopic representations.

Keywords: Saccade; Spatial stability; Spatiotopic representation.

Fig. 1

(A) Graphical illustration of setup used to study spatiotopy with adaptation. Subjects fixated loosely on the fixation point FP for 3000 ms, viewing an adapter grating patch (0.8 c/°, vignetted within Gaussian window of σ = 3.5°) tilted at +15° or −15°. The saccade target (ST) was then presented, to which subjects saccaded on extinction of the fixation point: either at onset of the saccade target, or 500 or 1000 ms later. (B) The test target came on 300 ms after extinction of the fixation point, always at least 30 ms after the eyes had landed. 300 ms after extinction of fixation point the test patch was presented for 51 ms in the spatiotopic, the retinotopic, or the control position, and subjects indicated the direction of tilt of the test patch. (C) Timecourse of events in a trial for all target preview durations. In the 0 ms target preview duration condition the saccade target appeared simultaneously with fixation point offset. In the other two conditions the saccade target was shown either 500 ms or 1000 ms before fixation point offset. (D) Tilt-aftereffect for the full-adaptation (blue), retinotopic (purple), spatiotopic (orange) and control (gray) conditions, as a function of preview duration of the saccade target, averaged over all subjects. Error bars represent ±1 SEM. (E) Normalized tilt-aftereffect results for the three eye-movement conditions (color-coding as for B). Aftereffect magnitude was divided by each subject’s full-adaptation magnitude, then averaged over subjects. Error bars represent ±1 SEM. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

Fig. 2

(A) Positions of the fixation point (FP) and the saccade target (T1) for the adaptation procedure. During execution of the saccade, the saccade target jumped in the outward direction. (B) Positions of the fixation point and the saccade targets (T1 and T2) in the “spatiotopic condition.” Subjects fixated on the fixation point until saccade target T1 appeared and then made a saccade to T1. From T1 they had to initiate a saccade to T2. (C) Effect of adaptation on the amplitude of the horizontal saccadic vector of the second saccade in the double-step sequence, expressed as the difference in average amplitude before and after adaptation. Error bars represent the standard error of the sample mean. Adaptation occurs in the spatiotopic (and full) conditions, but not in the retinotopic conditions. (D) Effect of head-turn on adaptation-induced horizontal saccade amplitude changes of the second saccade in the double-step sequence. Horizontal saccades were adapted with the head facing the screen center, and test trials were collected with the head turned 9_ leftward (purple) or rightward (orange). The adaptation remained spatiotopic with head turns. Error bars represent the standard error of the sample mean. (E) Mean shift in saccade landing position for the 3 overlap durations during adaptation in the spatiotopic condition. Shifts of saccades performed to targets presented for 50 ms are shown in blue, and shifts of saccades performed to targets presented for 500 ms are shown in red. Error bars represent SE. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)

Fig. 3

(A) Timecourse of events in the orthogonal double step task where the saccade targets T1 and T2 were presented either for 60 ms (upper panel) or 500 ms (lower panel). (B) Mislocalization of probe dots at onset of first saccade. The start of each arrow represents baseline localization from 100 ms before onset of the first saccade. The arrowhead indicates peak mislocalization at first saccade onset. Error bars indicate the horizontal and vertical SEM of the baseline and the peak mislocalization. (C) Spatial distribution of mislocalization at second saccade onset. Same conventions as (B). (D) Average compression magnitude (0 is maximum, 1 minimum) in trials with different presentation duration of the saccade targets T1 and T2 (between 60 and 500 ms). Data were averaged across subjects and trials. Errors are SE of the mean.

Fig. 4

(A) Experimental setup for saccade and fixation trials. The black circles indicates eye position. The red squares indicate targets for fixation or saccades. A trial started with subjects directing gaze to a fixation point. After 100 ms, the fixation point was turned off and subjects continued to maintain fixation on the blank screen. The saccade target T1 appeared 1000 ms later. In the saccadee trials subjects saccaded to the target on auditory cue 0–500 ms after saccade target onset. As soon as the saccade was detected, the saccade target was displaced either leftwards or rightwards (T2). At the end of the trial, the subject indicated the direction of the target displacement by key press. In the fixation trials subjects maintained fixation at the position of the fixation point for the entire trial. T1 was displayed for 0–900 ms, and followed by 60 ms of high-contrast mask (simulating the masking effect of the saccade). T2 was then displayed leftwards or rightwards of its original position and subjects reported the direction of the shift by key press. (B) Displacement thresholds (geometric mean of thresholds, calculated separately for all subjects) for the saccade (shown in red) and the fixation (shown in green) condition as a function of presentation duration. Saccade latencies were measured for all trials, and target presentation time binned into 100 ms bins. Error bars represent standard error across subjects. The continuous curves show exponential fits to the data. (For interpretation of the references to color in this figure legend, the reader is referred to the web version of the article.)