Spatiotopic coding and remapping in humans

Abstract

How our perceptual experience of the world remains stable and continuous in the face of continuous rapid eye movements still remains a mystery. This review discusses some recent progress towards understanding the neural and psychophysical processes that accompany these eye movements. We firstly report recent evidence from imaging studies in humans showing that many brain regions are tuned in spatiotopic coordinates, but only for items that are actively attended. We then describe a series of experiments measuring the spatial and temporal phenomena that occur around the time of saccades, and discuss how these could be related to visual stability. Finally, we introduce the concept of the spatio-temporal receptive field to describe the local spatiotopicity exhibited by many neurons when the eyes move.

Figure 1.

Integration of motion signals across saccades. (a) Illustration of experimental setup. Subjects fixated above a field of dynamic random noise. When cued they saccaded to a target below this field. (b) Timeline of stimuli presentation. For two brief (150 ms) moments, a proportion of the dots moved coherently, both leftwards or rightwards. One motion signal was presented before the saccade, the other after. Subjects had to identify the direction of motion in two-alternative forced choice. (c) Coherence sensitivity (inverse of minimum coherence to support reliable direction discrimination) as a function of separation of the two pulses. The dashed line shows the sensitivity for just a single motion signal. Presenting two stimuli with brief separation, either in fixation or straddling saccades, doubles the sensitivity, implying total integration. The integration continues for longer when interspersed with a saccade than when presented during fixation (filled symbols indicate fixation and open symbols indicate saccade).

Figure 2.

BOLD response amplitudes for area MT for one hemisphere of an example subject as a function of the spatiotopic stimulus coordinates (0 is screen centre), during (a) passive fixation and (b) the high-load attentional task. The responses are colour-coded by fixation (red −8°, black 0°, blue +8°: fixation indicated by the dotted coloured lines). In the passive viewing the responses at all three fixations line up well, consistent with spatiotopic selectivity; with foveal attention they are clearly displaced in the direction of gaze, and become retinotopic. The inserts at right show by colour-code how the ‘spatiotopicity index’ of voxels in the region is dependent on attention. During passive fixation, most of MT is blue (spatiotopic), but when the subject performed the attention-demanding foveal task these voxels became strongly retinotopic (red/yellow code). The index was similar to that used by Gardner et al. [31]. It is the difference of the squared residuals differences in response amplitude for the three fixation conditions when they are in a spatiotopic alignment (resid_S) and retinotopic alignment (resid_R), normalized by the sum of the squared residuals .The index is constrained between −1 (full spatiotopicity) and +1 (full retinotopicity).

Figure 3.

Spatial and temporal effects of saccades. (a) Perceived position of stimuli briefly flashed just before a saccadic eye movement. All stimuli are seen as displaced towards the saccadic target, resulting in a massive compression of space: all stimuli falling between −10 and +20° were seen at +7°. (b) Perceived time of stimuli presented around the time of saccades. Time perception is not veridical during saccades, but shows a similar compression towards saccadic landing. (c) Temporal against spatial mislocalization for experiments for trials in which subjects were required to localize objects in both space and in time. The errors correlate highly with each other (R² = 0.92).

Figure 4.

Saccadic compression of space and time at the time. (a) A number of bars, varying randomly from zero to four, were presented at random positions around saccadic target, and the subject reported how many she saw. Open symbols refer to trials when there were four bars present, triangles to trials with one bar and filled circles to catch trials with none. Zero and one were reported correctly, but four bars were all compressed together as one when presented near saccadic onset. (b) Perceived duration as a function of presentation time, relative to saccadic onset. The stimulus pair were separated by 100 ms, but perceived as separated by 50 ms when presented near saccadic onset.

Figure 5.

Localization of visual and auditory stimuli, during fixation and saccades. (a) Subjects were required to report in forced choice which of two bars seems to be more rightward, one presented in fixation, the other peri-saccadically (open squares). Filled grey squares show the results when both were presented in fixation. During saccades the curve is displaced—reflecting a bias in judgements—and broader—reflecting a reduction in precision. (b) Results for localization of visual bars (open squares), auditory click (open circles) and audio-visual bar-clicks (filled triangles). The audio-visual results show both less bias and improved precision, suggesting that during saccades auditory signals are as reliable as visual signals.

Figure 6.

Spatio-temporal transformation of RFs at the time of saccades. (a,b) Illustration of ‘classical’ RF during fixation (a, light blue) and displacement to ‘future’ RF just before saccades (b, pink). (c,d) Example responses of an FEF neuron to stimuli presented in the classical RF (light blue), future RF (pink) or an irrelevant position (grey), either during fixation (c) or just prior to the saccade (d). The responses to the future RF are delayed with respect to the classic RF. (e) Cartoon drawn from data of Wang et al. [99] reporting the response of an LIP neuron. All responses are aligned to the saccadic onset (bottom trace in black, illustrated in external space), and sorted by time from saccadic onset (shown in the ordinate). Irrespective of the time of stimulation, all spikes tend to arrive at the same time. (f) Spatio-temporal RF of the neuron (in retinal space), defined as the region of confusion in space–time that gives rise to the same spiking pattern. As the eye movement (illustrated by the icon above, in external coordinates) changes the retinal position, a transient flash delivered to the pink circle (future RF) before the saccade will be fused with a flash delivered to the light-blue circle (classical RF) after the saccade by a neuron with the oriented RF in space–time as illustrated by the colour-coded plot. This spatio-temporal RF is oriented in space–time along the trajectory of the retinal motion created by the saccade, and, therefore, effectively stabilizes transiently the image on the retina.