The neural basis of depth perception from motion parallax

Abstract

In addition to depth cues afforded by binocular vision, the brain processes relative motion signals to perceive depth. When an observer translates relative to their visual environment, the relative motion of objects at different distances (motion parallax) provides a powerful cue to three-dimensional scene structure. Although perception of depth based on motion parallax has been studied extensively in humans, relatively little is known regarding the neural basis of this visual capability. We review recent advances in elucidating the neural mechanisms for representing depth-sign (near versus far) from motion parallax. We examine a potential neural substrate in the middle temporal visual area for depth perception based on motion parallax, and we explore the nature of the signals that provide critical inputs for disambiguating depth-sign.This article is part of the themed issue 'Vision in our three-dimensional world'.

Keywords: depth; motion parallax; neural computation.

Figure 1.

Similarity between motion parallax and binocular disparity as depth cues. (a) Motion parallax. If the head translates rightward, the image of a far object (open symbol, top) moves on the retina. If the eye moves through one inter-ocular distance, the position change on the retina due to motion parallax is equivalent to the object's binocular disparity (as shown in panel b). Hence depth from motion parallax is often expressed in units of equivalent disparity. (b) Binocular disparity. Points falling along the geometric horopter, or Vieth-Muller circle (curved line), have zero binocular disparity. Here, the far object projects to disparate points in the retinal image for the two eyes (bottom). The binocular disparity shown here is equal to the change in position of the monocular image in a.

Figure 2.

Schematic of motion parallax and stimulus design used by Nadler et al. [35] to establish neural correlates of depth from motion parallax. (a) As the head moves to the right, the image of a near object moves leftward, while the image of a far object moves rightward. (b) The opposite occurs during leftward head movement. Without pictorial depth cues, an extra-retinal signal is needed to determine depth-sign. (c) Random-dot stimuli were scaled so that size and density were identical across simulated depths. Three depths—far (+1°), near (−1°) and zero—are illustrated. (d) Animals were passively translated laterally using a motion platform. Movement followed one cycle of a modified sinusoid in the frontoparallel plane, and animals were trained to maintain fixation on a world-fixed target. Thick black and grey curves represent the head movement trajectory for two possible starting phases. Thin curves represent average eye position and velocity traces for a single session, in equivalent stimulus units. (Reproduced from Nadler et al. [35].)

Figure 3.

Example neurons illustrating that signals related to eye movements, not head movements, disambiguate depth-sign tuning in area MT. In each panel, depth-tuning curves are shown for each of four stimulus conditions: motion parallax (black), retinal motion (blue), head only (green) and eye only (orange). Average firing rates are plotted as a function of simulated depth. Error bars represent s.e.m. (a) An example neuron that prefers near depths in the motion parallax and eye only conditions, but has no depth-sign preference (U-shaped tuning) in the retinal motion and head only conditions. (b) Analogous results from another example neuron that prefers far depths in the motion parallax and eye only conditions. (Adapted from Nadler et al. [43].)

Figure 4.

Eye movement signals, not vestibular translation signals, produce depth-sign tuning in MT neurons. This scatter plot compares depth-sign discrimination index (DSDI) values for the eye only condition (filled symbols) and the head only condition (open symbols) to DSDI values obtained in the motion parallax condition (79 neurons from two monkeys; M1: circles, M2: triangles). DSDI is a measure of depth-sign preference, such that negative values indicate neurons that prefer near depths and positive values indicate neurons that prefer far depths. Only DSDIs obtained in the eye only condition are correlated with those obtained in the motion parallax condition. (Adapted from Nadler et al. [43].)

Figure 5.

Schematic of variables involved in the motion-pursuit law. The observer's eye translates rightward while maintaining fixation on the point F, which is located at a viewing distance given by f. A point of interest, D, has a depth, d, relative to the plane of fixation. θ(t) denotes the time-varying angle subtended by the point D relative to the fovea. α(t) represents the time-varying orientation of the eye relative to the scene. (Reproduced from Nawrot & Stroyan [27].)

Figure 6.

Stimulus conditions used in the dynamic perspective experiment of Kim et al. [58] and results from an example neuron. (a) Frontal views for each experimental condition. In the motion parallax condition, animals experience full-body translation and make counteractive eye movements to maintain fixation on a world-fixed target (yellow cross). In the retinal motion condition, the animal's head and eyes are stationary, but visual stimuli replicate the image motion experienced in the motion parallax condition. The dynamic perspective condition is the same as the retinal motion condition except that a three-dimensional cloud of background dots was added to the display. Background dots near the receptive field (RF) were masked. (b) For each stimulus condition, depth-tuning curves are shown for an example MT neuron. Tuning in the retinal motion condition is symmetrical around zero depth (blue, DSDI = –0.09), whereas tuning curves show a clear preference for near depths in the motion parallax (black, DSDI = –0.80) and dynamic perspective (magenta, DSDI = –0.67) conditions. Error bars represent s.e.m. (Adapted from Kim et al. [58].)

Figure 7.

Comparison of depth-tuning from motion parallax and disparity. (a) Depth-tuning curves are shown, for an example MT neuron, for the retinal motion (blue), motion parallax (black), binocular disparity (green) and combined (orange) conditions. In the binocular disparity condition, depth was cued solely by disparity, without motion parallax. The combined condition was similar to the motion parallax condition except that dots were presented dichoptically. Each tuning curve plots mean firing rate (averaged across the duration of the trial as well as the two starting phases of motion) against simulated depth of the stimulus, and error bars denote s.e.m. This neuron has consistent depth-sign preferences for disparity and motion parallax. (b) Another example neuron, with opposite depth-sign preferences for disparity and motion parallax. Format as in a. (c) The depth-sign discrimination index (DSDI) for the binocular disparity (BD) condition (ordinate) is plotted against the DSDI value for the motion parallax (MP) condition (abscissa) (N = 134). Filled black symbols indicate DSDI values that are significantly different from zero (p < 0.05, permutation test) in both the motion parallax and binocular disparity conditions. Filled red symbols indicate DSDI values that are significantly different from zero in the motion parallax condition, but not in the binocular disparity condition. Open black circles indicate DSDI values that are significantly different from zero in the binocular disparity condition, but not in the motion parallax condition. Finally, open red circles indicate DSDI values that are not significantly different from zero in either the binocular disparity or motion parallax conditions. (Adapted from Nadler et al. [70].)

Figure 8.

Schematic of the local discrepancy between disparity and motion cues that can arise for a moving object in the world. In this illustration, an observer maintains visual fixation on the traffic light while moving their head to the right. In this case, all stationary objects in the scene have an image velocity that is determined by their three-dimensional location in the scene (as specified by binocular disparity, for example) and the movement of the observer's head. Stationary near objects move leftward in the image, whereas stationary far objects move rightward in the image. However, a moving object (the car) creates local retinal image motion that is not consistent with that expected from the binocular disparity of the object and the movement of the observer. This local discrepancy between disparity and retinal image motion might be sensed by the relative activity of congruent and opposite cells in area MT. (Adapted from Nadler et al. [70].)