Binocular Mechanisms of 3D Motion Processing

Abstract

The visual system must recover important properties of the external environment if its host is to survive. Because the retinae are effectively two-dimensional but the world is three-dimensional (3D), the patterns of stimulation both within and across the eyes must be used to infer the distal stimulus-the environment-in all three dimensions. Moreover, animals and elements in the environment move, which means the input contains rich temporal information. Here, in addition to reviewing the literature, we discuss how and why prior work has focused on purported isolated systems (e.g., stereopsis) or cues (e.g., horizontal disparity) that do not necessarily map elegantly on to the computations and complex patterns of stimulation that arise when visual systems operate within the real world. We thus also introduce the binoptic flow field (BFF) as a description of the 3D motion information available in realistic environments, which can foster the use of ecologically valid yet well-controlled stimuli. Further, it can help clarify how future studies can more directly focus on the computations and stimulus properties the visual system might use to support perception and behavior in a dynamic 3D world.

Keywords: binocular; depth; disparity; geometry; motion; optic flow.

Figure 1

The binoptic flow field (BFF). (a) Photograph (taken from the Scheiner/Descartes perspective) of the projection of a BFF generated by points on a frontoparallel plane on the retina of a realistic model eye in the left (left, green-tinted projections) and right (right, red-tinted projections) eye positions. Fixation was on the center of the stimulus at its initial position. The visual system faces the important challenge of converting this proximal stimulus into veridical estimates of self-motion and object motion in the external world. (b) An overview of a computer simulation of a situation similar to that shown in panel a. The eyeballs are 3.25 cm to the left and right of the origin, and the visual axes of the right and left eyes are shown by the red and green lines, respectively. The yellow vectors show the motion of points fixed to a plane moving along the z axis. (c) The retinal projections of the points in panel b shown at five uniformly spaced times during the motion. The eyeballs were modeled as 2.5-mm spheres with centers of rotation at the centers of the spheres and with a 16-mm focal-length thin lens. Fixation was on the center of the stimulus at time zero. The eyeballs and projections are rendered from directly behind them—the Scheiner/Descartes view as in panel a. (d) The retinal projections of the motion vectors shown in panel b, after rotating and shifting to align idealized corresponding retinal points.

Figure 2

Binoptic flow fields (BFFs) generated as in Figure 1b–d. The top subpanels show overviews of the three situations: (a) an observer walking over a ground plane, (b) an observer walking through a 2-m diameter tunnel, and (c) an observer walking to a door opening into a 3-m deep room. The midpoint between the eyes is at (0, 0, 0), and fixation is at eye height at the farthest point in each environment. The visual axes of the right eye are shown by the red lines. In panel c, the front wall itself—the wall containing the open door—is not shown for clarity but the corresponding motion vectors are; the motion vectors shown on the floor and back wall are those visible through the doorway. The bottom subpanels show the corresponding BFFs; note that, because these are retinal projections, the lower visual fields correspond to positive y values. In panel c (bottom), the top of the doorway is quite obvious at around y = −1 mm. We think that these flow fields make it rather clear that describing depth motion in terms of either changing horizontal disparities or horizontal velocity difference is somewhat impoverished.

Figure 3

Response of a simulated middle temporal area (MT) neuron to real-world object motion: unequal tuning for rightward motion in each eye. A simple instantiation of an interocular velocity difference (IOVD) neuron is composed of a different speed preference in the left and right eye (10°/s and 1°/s, respectively), which corresponds to a 3D motion trajectory slightly to the right of the observer’s head. (a–c) Each panel simulates a fixed object speed equivalent to that of, for example, (a) an approaching toddler, (b) a brisk walk, and (c) a ball tossed from 10 m away (50, 200, and 1,200 cm/s, respectively). Response (spikes per second, colormap) is plotted as a function of viewing distance (y axis) and simulated 3D motion trajectory (x axis; within the horizontal plane). Viewing distances range from a few centimeters (3.25 cm; one-half interpupillary distance) to 5 m. Contours of equal velocity ratio (white dashed lines) are independent of stimulus speed, and thus are shown only in panel a. Black dashed lines indicate the particular contour corresponding to the binocular tuning of the simulated neuron (10:1). Equivelocity contours (left eye, green; right eye, red) for select monocular velocities are overlaid on the response surface and highlight the influence of viewing distance on projected monocular velocity. The open circles show the intersection of the two contours corresponding to the neuron’s preferred velocities in each eye (and hence necessarily also lie on the preferred ratio contour) and thus correspond to the peak response of the cell. (d) Simulated response to each object speed on the preferred ratio contour (black dashed lines in panels a–c) as a function of viewing distance. Additional information from the binoptic flow field is necessary to disambiguate responses based on IOVD information alone. Abbreviations: A, away; L, left; R, right; T, toward.

Figure 4

Response of simulated middle temporal area (MT) neurons to real-world object motion: equal but opposite tuning for motion in each eye. (a) Five example objects are located on a circle around an observer located at (0, 0), with the eyes marked by asterisks. For each object, a cell’s response is shown in polar coordinates for motion to all directions. (b–d) In the left column, the stimulus and response are plotted as a function of environmental direction (icons around the polar plot correspond to icons on the x axes). (e–g) In the right column, these are plotted as a function of egocentric direction, the space in which a theoretical collision-detector such as our simulated cell would operate. The rows plot (b, e) retinal velocities, (c, f) IOVDs, and (d, g) the responses of the cells. The direction of the cells’ peak response is largely invariant to stimulus eccentricity. Abbreviations: A, away; IOVD, interocular velocity difference; IOVO, interocular velocity opponency; L, left; OV, ocular velocity; R, right; T, toward.