Temporal Coding of Visual Space

Abstract

Establishing a representation of space is a major goal of sensory systems. Spatial information, however, is not always explicit in the incoming sensory signals. In most modalities it needs to be actively extracted from cues embedded in the temporal flow of receptor activation. Vision, on the other hand, starts with a sophisticated optical imaging system that explicitly preserves spatial information on the retina. This may lead to the assumption that vision is predominantly a spatial process: all that is needed is to transmit the retinal image to the cortex, like uploading a digital photograph, to establish a spatial map of the world. However, this deceptively simple analogy is inconsistent with theoretical models and experiments that study visual processing in the context of normal motor behavior. We argue here that, as with other senses, vision relies heavily on temporal strategies and temporal neural codes to extract and represent spatial information.

Keywords: eye movements; microsaccade; ocular drift; retina; saccade; space perception; temporal processing; visual fixation; visual system.

Figure 1.. Space-time consequences of eye movements.

(A) Eye movements transform spatial patterns of luminance into temporal modulations impinging on the retina. In the frequency domain, this transformation redistributes the spatial power of the stimulus from 0 Hz (green line) across nonzero temporal frequencies (blue surface) in a way that depends on the type of eye movements. (B) Spatiotemporal power redistributions resulting from saccades (red curve) and fixational drifts (blue). Each curve represents the average fraction of stimulus power that eye movements make available at 10 Hz. (C) Spectral density of the retinal input during viewing of natural scenes. Data represent average distributions at 10 Hz. The power distributions made available by saccades and drift drive cell responses at different times during the course of post-saccadic fixation. (D-E) Drift and saccade transients contribute to different ranges of visual sensitivity. Elimination of drift and saccade transients (D) impairs contrast sensitivity at 10 cycles/deg (high), but has little effect at 1 cycle/deg (low). The opposite effect occurs with elimination of saccade transients (E). (F) Time course of visual sensitivity during natural post-saccadic fixation. Following a saccade, contrast sensitivity continues to improve at 10 cycles/deg but not at 1 cycle/deg. Data adapted from [58]. (*) indicates p<0.03.

Figure I (Box 1).. Normal eye movements.

(A) An oculomotor trace (yellow) superimposed on the observed scene. Each fixational pause is represented by a circle with radius proportional to its duration. The zoomed-in panel (top right) shows fixational eye movements. Both microsaccades (yellow segments) and drifts (orange) are visible. (B) Horizontal and vertical eye displacements in a portion of the trace. The shaded red regions indicate microsaccades. (C) Luminance modulations experienced by one retinal receptor, as the eye changes fixation via a saccade.

Figure I (Box 2).

Boundaries for spatial coding of vernier offsets.

Figure I (Box 3):. Spatio-temporal sensitivity profile of a unit selective to rightward motion.

Red indicates positive response, blue negative.