Consequences of the Oculomotor Cycle for the Dynamics of Perception

Abstract

Much evidence indicates that humans and other species process large-scale visual information before fine spatial detail. Neurophysiological data obtained with paralyzed eyes suggest that this coarse-to-fine sequence results from spatiotemporal filtering by neurons in the early visual pathway. However, the eyes are normally never stationary: rapid gaze shifts (saccades) incessantly alternate with slow fixational movements. To investigate the consequences of this oculomotor cycle on the dynamics of perception, we combined spectral analysis of visual input signals, neural modeling, and gaze-contingent control of retinal stimulation in humans. We show that the saccade/fixation cycle reformats the flow impinging on the retina in a way that initiates coarse-to-fine processing at each fixation. This finding reveals that the visual system uses oculomotor-induced temporal modulations to sequentially encode different spatial components and suggests that, rather than initiating coarse-to-fine processing, spatiotemporal coupling in the early visual pathway builds on the information dynamics of the oculomotor cycle.

Keywords: coarse-to-fine; contrast sensitivity; fixational eye movements; magnocellular; microsaccade; ocular drift; parvocellular; retina; saccade; spatial vision.

Figure 1. Natural input to the retina and predicted neural dynamics

(A) Normal alternation between saccades (yellow segments) and periods of fixational eye movements (red circles) during viewing of natural scenes (image size: 17° × 12. 8°). The size of each circle represents fixation duration. A portion of the eye movement trace (orange) is expanded on the right panel (top), together with the luminance signal experienced by a retinal receptor (bottom). Eye movements continually modulate the input flow onto the retina, effectively redistributing spatial information in the joint space-time domain. (B) Mean instantaneous speed of the retinal image (note log scale) and (C) mean rate of change in the input luminance experienced by a foveal receptor during a fixation-saccade-fixation sequence. Time zero marks saccade end. (D–F) Neural modeling. (D) The responses of model V1 simple cells were simulated as their receptive fields moved following recorded oculomotor traces. (E) Models consisted of rectified filters with separable spatial (top) and temporal (bottom) kernels. (F) Responses of neurons tuned to low (1 cycles/deg, blue line) and high spatial frequency (10 cycles/deg, red line). Solid lines represent medians of activity. Shaded regions enclose the range between the first and third quartile.

Figure 2. Power spectra of the retinal input signals resulting from different types of eye movements

(A, B) Eye movements transform spatial information in the scene into spatiotemporal modulations on the retina, redistributing the power at 0 Hz of a static scene (A) across non-zero temporal frequencies (B). The characteristics of this transformation depend on the type of eye movements. (C, D) Spectral density of the visual input present during fixational drift. Both the full spatiotemporal distribution (C) and sections at individual temporal frequencies (D) are shown. Sections in D are compared to the ideal 1/k² power spectrum of natural images (Natural). During viewing of natural scenes, fixational drift equalizes power across a broad range of spatial frequencies. (E, F) Spectral density of the visual input given by saccades. Unlike ocular drift, saccades yield modulations that follow the spectral density of natural images over a wide range of temporal frequencies.

Figure 3. Predicted dynamics of contrast sensitivity

(A) The responses of V1 neurons were modeled in a detection task with gratings embedded in a naturalistic noise field (top). Models were tuned to the grating’s orientation and spatial frequency (either 1 or 10 cycles/deg), and their receptive fields translated following sequences of recorded eye movements (two fixations separated by a saccade; bottom). (B) At each time during post-saccadic fixation, a standard decision-making model determined the presence or absence of the grating based on the cumulative neural response (averaged across all simulated cells) from fixation onset (η(t); top). The contrast yielding a hit rate of 0.75 and false alarm rate of 0.25 (d′ = 1.35) was selected as threshold (bottom). (C) Average responses with or without the grating at 1 (top) and 10 cycles/deg (bottom). Shaded regions mark the post-saccadic period of fixation considered by the model in B. The darker shade represents the period of early fixation in which the preceding saccade influences neural responses. Note how responses to the high spatial frequency target remain well-separated from the noise response not only in early fixation, but also in late fixation. (D) Schematic representation of the impact of different types of eye movements on visual detection. For any given contrast, the separation between the distributions of cell responses in the presence and absence of the grating transiently increases following saccades at low spatial frequencies, but remains more similar throughout the course of fixation at high spatial frequencies. (E) Model predictions. With a 1 cycle/deg grating, contrast sensitivity does not increase beyond the early-fixation level provided by the saccade transient. Sensitivity to 10 cycles/deg improves more gradually, as it also weights the amplified response that persists during late fixation because ocular drift.

Figure 4. Post-saccadic dynamics of contrast sensitivity

(A–B) Experimental procedure. (A) Subjects reported the orientation (±45°) of a grating at either 1 (low frequency; top) or 10 cycles/deg (high frequency; bottom) embedded in a full-screen naturalistic noise field (1/k² spectrum; shaded triangle). (B) Subjects initially fixated ~7° away and performed a saccade toward the future grating location at the appearance of a cue. The stimulus appeared during the saccade and was displayed for either 100 or 800 ms following fixationonset. (C) Contrast sensitivity (N=4). Sensitivity increased with longer stimulus exposure for high but not low spatial frequencies. (D) Relative change in sensitivity with exposure duration. In both C and D, colored symbols are individual subject data, black symbols are group means, and error bars represent SEM. Asterisks mark statistically significant differences (p < 0.03; paired two-tailed t-test).

Figure 5. Perceptual consequence of fixational drift

(A) Experimental procedure. Following the initial saccade, stimuli were either displayed at a fixed location on the monitor (Normal) or at a fixed location on the retina (Stabilized). In this latter condition, stimuli moved on the screen under real-time control to compensate for the subject’s eye movements. (B) Experimental results. Contrast sensitivity in the two viewing conditions at 1 (low) and 10 cycles/deg (high). Eliminating drift modulations impairs sensitivity at high (left panel), but not low (right panel), spatial frequencies. Each graph shows data from two observers. Error bars represent 95% confidence intervals. (*) p = 0.0035; (**) p=0.0005; parametric bootstrap test.

Figure 6. Perceptual consequences of saccade transients

(A) Experimental procedure. The stimulus was either displayed during the saccade (Normal condition) or gradually increased in contrast after saccade end (No-transient condition). The graphs show the dynamics of stimulus contrast in the two conditions. (B) Experimental results. Contrast sensitivity in the presence and absence of saccade transients for gratings at 1 and 10 cycles/deg. Small colored symbols are individual subject data, large gray and black symbols are group means. Error bars represent SEM. (*) p <0.02; paired two-tailed t-test; N=4.