Curvature of visual space under vertical eye rotation: implications for spatial vision and visuomotor control

Abstract

Most models of spatial vision and visuomotor control reconstruct visual space by adding a vector representing the site of retinal stimulation to another vector representing gaze angle. However, this scheme fails to account for the curvatures in retinal projection produced by rotatory displacements in eye orientation. In particular, our simulations demonstrate that even simple vertical eye rotation changes the curvature of horizontal retinal projections with respect to eye-fixed retinal landmarks. We confirmed the existence of such curvatures by measuring target direction in eye coordinates in which the retinotopic representation of horizontally displaced targets curved obliquely as a function of vertical eye orientation. We then asked subjects to point (open loop) toward briefly flashed targets at various points along these lines of curvature. The vector-addition model predicted errors in pointing trajectory as a function of eye orientation. In contrast, with only minor exceptions, actual subjects showed no such errors, showing a complete neural compensation for the eye position-dependent geometry of retinal curvatures. Rather than bolstering the traditional model with additional corrective mechanisms for these nonlinear effects, we suggest that the complete geometry of retinal projection can be decoded through a single multiplicative comparison with three-dimensional eye orientation. Moreover, because the visuomotor transformation for pointing involves specific parietal and frontal cortical processes, our experiment implicates specific regions of cortex in such nonlinear transformations.

Fig. 1.

Simulated eye position-dependent geometry of retinal stimulation. A, Stimulus array viewed from a distance, showing the objective locations of its components. Simulated target pairs are located on five horizontal semicircles centered around the eye, elevated (in terms of gaze angle) at 30° up, 15° up, 0°, 15° down, and 30° down. The green sphere andblue sphere indicate two possible fixation points, with two targets (green and blue squares) placed 90° to the right from the perspective of the eye (which currently points to center). B, Close up view of the semitransparent eye from behind while it looks toward the central blue sphere. The optically inverted projections of the stimulus lines onto the retina are visible. C, Similar view with the eye fixating the central target on the topmost line, as in A. D, Same situation asC but now viewed from an eye-fixed perspective, looking down the line of gaze toward the top stimulus line. This simulation can be viewed as an interactive animation athttp://www.physiology.uwo.ca/LLConsequencesWeb/index.htm.

Fig. 2.

Experimental pointing paradigms. A,B, Examples of four eye and arm trajectories recorded during each of the two pointing paradigms, plotted as a function of time. F, Duration of fixation target. T, Duration of pointing target. In these particular cases,F was 15° up and T was 60° to its right. ▪, Horizontal arm orientation. ■, Vertical arm orientation.Thick lines, Horizontal eye orientation. Thin lines, Vertical eye orientation. A, The double-point paradigm. Subject first pointed toward Fand then continued to fixate on F while pointing towardT. B, The single-point paradigm. Subject pointed directly to T while maintaining fixation onF. Subjects consistently showed a transient postmovement downward drift of the arm resembling saccadic pulse-step mismatch at all target levels in both paradigms. C,D, Corresponding 2-D trajectories of upper arm orientation for the same movements.

Fig. 3.

Stimulus locations in spatial and retinal frames in one typical subject. A, Target locations in space coordinates, computed from eye position signals recorded while subjects fixated each target. ●, Target location used for ocular fixation and initial pointing direction. ○, Target location used for final pointing direction. Dashed lines indicate the pairing of fixation and pointing targets during experiments. In this and subsequent figures, angular directions are represented with the use of unit-length vectors aligned with the pointing direction and projected onto a frontal plane (Klier and Crawford, 1998), such that the scale follows a sine function and the locations of oblique targets appear to be slightly distorted compared with their locations in translational space. B, Target directions (○) in retinal coordinates (right eye). The horizontal and vertical axes are the flat projections of the orange retinal meridian in Figure 1,B and D, viewed from behind the eye, but the optical inversion is dispensed with so that rightward vectors indicate rightward targets, etc. To derive these vectors, the original direction vector for each rightward target (○) in Awas rotated by the inverse of the average measured 3-D eye orientation vector while the subject fixated (●) (Klier and Crawford, 1998). Thus, the fixation target (●) now always corresponds to the fovea, and the horizontal coordinate axis corresponds to horizontal retinal meridian (defined here as the retinal arc intersected by the horizontal plane passing through the center of the eye when gaze is directed straight ahead). Note that the pattern of stimulation was symmetric about the horizontal meridian in this particular subject, whose Listing's plane of 3-D eye orientation vectors (C) happened to align closely with the spatial frontal plane. However, in subjects with tilted Listing's planes (i.e., in which ocular torsion was a function of gaze angle), the pattern was predictably skewed either upward or downward, as described previously (Klier and Crawford, 1998). This is one reason why it is necessary to use 3-D eye orientation to compute retinocentric target vectors.

Fig. 4.

Predicted and actual pointing trajectories from the double-point paradigm in two subjects. A, Predicted responses. ●, Average initial 2-D arm position. ⊗, Final positions predicted by traditional models that compute the target direction based on addition of the current gaze direction with the retinal vector (Hallet and Lightstone, 1976; Zee et al., 1976; Mays and Sparks, 1980;Howard, 1982; Zipser and Andersen, 1988; Flanders et al., 1992; Miller, 1996; Bockisch and Miller, 1999) or in this task if the arm were simply displaced in the direction coded by the retina. Gray wedges, Predicted angle of error from the (due rightward) ideal trajectory. B, Actual responses. Corresponding actual angular arm trajectories (♦) for five movements at each height, done in complete darkness to a previously flashed target. ○, Control pointing directions with full visual feedback of arm and target.C, D, Similar data for the subject whose arm trajectories came closest to following the predictions of a linear model. Note that the predicted pattern of error for each subject was not generally identical because it was also influenced by the orientation of Listing's plane, which varies between subjects (Klier and Crawford, 1998). Also note that the arm angles were slightly different than those of the eye (Fig. 3A) because they do not share the same center of rotation, but otherwise this task provides motor invariance at different vertical levels in terms of the axis of arm rotation during pointing (Hore et al., 1992; Miller et al., 1992).

Fig. 5.

Performance of one typical subject in the single-point paradigm. ●, Measured fixation directions. ■, Desired pointing direction to T determined from controls with full visual feedback. ♦, Actual final pointing directions in the absence of visual feedback. Horizontal lines connect corresponding fixation and pointing data for illustrative purposes only; they do not represent trajectories or any other meaningful variable. Data are arranged (A–F) according to the horizontal locations of the fixation and pointing targets for clarity, but targets were randomized during the experiment.

Fig. 6.

Summary of actual (vertical axis) versus predicted (horizontal axis) pointing errors in the single-point paradigm.A, Individual data points for each of 24 stimulus pairs, each averaged across five movements, for one typical subject. The average vertical component of predicted angular error (computed from initial 3-D fixation positions and target vector measurements) is plotted along the horizontal axis. This signifies the constant error that would be made if the system failed to account for the measured vertical curvatures in retinal location induced as a function of initial eye orientation. Average angular errors in the actual responses (relative to ideal responses with visual feedback) are plotted along the vertical axis. Vertical error bars show SD across pointing trials for each target (horizontal variance was too small for graphic display). Also shown is a line fit by regression to the average points.B, Solid lines, Similar lines of regression fit for all six subjects. Hatched lines, Alignment of the data parallel to the horizontal axis represents complete rotational compensation for eye orientation, whereas alignment of the data parallel to the slope of unity (dotted lines) represents zero rotational compensation for eye orientation.

Fig. 7.

Retinal projection geometry of straight (in the Euclidean sense) horizontal lines in a fronto-parallel plane.A, Lines viewed from behind a semi-transparent “head” indicating subject's position. A horizontal pair of targets is placed straight ahead (●) and 45° right (▪), with a similar pair (○, ■) at 45° angle up (gaze angle). B, Projections of lines and targets (same symbols) onto retina, as viewed from behind. Gray disk, Foveal region. Thick line, Eye-fixed great circle through the fovea that defines the horizontal retinal meridian. Note that the retinal projections of the Euclidean lines resemble nonparallel lines of longitude (except horizontally arranged). C, Projections of the same targets (minus irrelevant lines) onto the retina, viewed from the same space-fixed perspective but now with gaze rotated 45° upward so that the upper target (▪) stimulates the fovea (which, being at the back of the eye, is rotated down). Note that the current line of regard again falls on the horizontal retinal meridian. D, Same retinal projection pattern as C but viewed from an eye-fixed perspective, along the visual axis. Note that the target (■) that was up and right in space coordinates now stimulates a retinal point signifying purely rightward displacement in retinal coordinates. However, for the arm to point accurately from ○ to ■ in our double-point paradigm, it would have to follow an oblique rightward–downward trajectory, again requiring a multiplicative reference frame transformation.