Motion parallax is computed in the updating of human spatial memory

Abstract

As we move through space, stationary objects around us show motion parallax: their directions relative to us change at different rates, depending on their distance. Does the brain incorporate parallax when it updates its stored representations of space? We had subjects fixate a distant target and then we flashed lights, at different distances, onto the retinal periphery. Subjects translated sideways while keeping their gaze on the distant target, and then they looked to the remembered location of the flash. Their responses corrected almost perfectly for parallax: they turned their eyes farther for nearer targets, in the predicted nonlinear patterns. Computer simulations suggest a neural mechanism in which feedback about self-motion updates remembered locations of objects within an internal map of three-dimensional visual space.

Figure 1.

Is motion parallax incorporated in the updating of spatial memory? A, When an observer translates sideways, a near object moves through a larger visual angle than a far object because of parallax. B, Our subjects foveated a far central target 2 m away. A near target flashed at location T, although subjects typically mislocalized it at point T_p. C, The subject translated the head rightward, still fixating the far target, then looked to the remembered location of the near, flashed target. If the subject failed to compute parallax, his or her final gaze would be directed as indicated by the dotted lines. However, in fact, subjects looked to the remembered location (indicated by thick lines), suggesting a nearly perfect updating of the internal representation of space during the head translation.

Figure 2.

A typical trial illustrating the translational updating paradigm. A, The translational motion of the eye in space (in centimeters). B, The pointing direction of the eyes (version component in degrees). C, The movement of the eyes in depth (vergence in degrees). Thin horizontal lines indicate the direction and depth of the target relative to the eyes after the motion. Filled bars mark the durations of the fixation target (F), the nose target (NT), and the flashed target (T). Vertical dashed lines mark the period over which the saccadic updating response was taken. See Materials and Methods for additional explanation.

Figure 3.

Characteristics of target localization. Binocular fixation direction versus target direction (A, C) and binocular fixation depth versus target depth (B, D) are shown. A, B, Raw data of one subject together with the fitted regression line. C, D, The regression lines of all subjects (gray lines), together with the average across all subjects (black lines).

Figure 4.

Spatial updating performance. Direction (A, C) and depth (B, D) of binocular fixation versus the values that would be predicted if the subject perfectly updated the perceived location of the target are shown. A, B, One subject's performance, together with the fitted regression line. C, D, Regression lines of all subjects (gray lines), together with the average across all subjects (black lines).

Figure 5.

Translational-depth geometry. A, Geometry of translational updating. Target angle (φ) and target distance (D) after translation depend on four parameters: the initial distance of the target from the eyes (d), its direction relative to the eyes (θ), the translation of the eyes (T), and the direction of eye translation (τ). The relationships are given by Equations 1 and 2. B, C, The average amount ± SE of updating (θ - φ) of six subjects (data binned) for updating targets at initial direction (θ) of 30°. They show the same nonlinear patterns as perfect updating (lines).

Figure 6.

Updating gains of all subjects. Translational updating works better for eye translations attributable to head translations (A) than for eye translations resulting from head rotations (B). A gain of 1 indicates perfect updating (shown by a horizontal dashed line). Error bars indicate SDs.

Figure 7.

Retinotopic remapping. A, A target is selected from an eye-centered target map ( ), and its retinal error is passed to a motion generator that can rotate the eyes and head as well as translate the head. The motion generator can also be driven using voluntary commands. In generating the motion, it provides feedback signals about eye and head motion, as given by (the rotary velocity of the eyes in space, ) and (the translational velocity of the eyes, , with x indicating the vector cross-product). Both signals, computed in eye coordinates, are used to update the retinotopic target map. Updating circuits use local information about target depth and direction (as stored in the target map) to determine how that same representation is remapped during the motion. B, The derivation of the updating equation (Eq. 3). If the eyes are rotating, the target moves in opposite direction relative to the eyes, according to . Likewise, when the eyes are translating, the target moves with opposite velocity relative to the eyes, as given by . Combining the two equations yields Equation 3.

Figure 8.

Predictions of the model of remapping in the collicular map during eye translations induced by head translations. A, Two space-fixed targets, a (at depth 25 cm) and b (at depth 100 cm), are flashed when the head is in a rightward position (10 cm right from midline). Both stimulate the right side of the retina and are represented, therefore, on the right superior colliculus (SC). However, after the head translates into a leftward position (10 cm, left), the remembered targets are now to the left relative to the retina. In the collicular map, then, their representations must cross the midline, from the right to the left SC. However, for each target, the amount of remapping depends on the target depth: target a, closer to the eye, requires more remapping (a longer trajectory on the map) than target b, which is at farther distance from the eye. B, During translations, eye-centered remapping requires target representations to move at different speeds across the SC map, depending on the depth of each target.