Beyond the labeled line: variation in visual reference frames from intraparietal cortex to frontal eye fields and the superior colliculus

Abstract

We accurately perceive the visual scene despite moving our eyes ~3 times per second, an ability that requires incorporation of eye position and retinal information. In this study, we assessed how this neural computation unfolds across three interconnected structures: frontal eye fields (FEF), intraparietal cortex (LIP/MIP), and the superior colliculus (SC). Single-unit activity was assessed in head-restrained monkeys performing visually guided saccades from different initial fixations. As previously shown, the receptive fields of most LIP/MIP neurons shifted to novel positions on the retina for each eye position, and these locations were not clearly related to each other in either eye- or head-centered coordinates (defined as hybrid coordinates). In contrast, the receptive fields of most SC neurons were stable in eye-centered coordinates. In FEF, visual signals were intermediate between those patterns: around 60% were eye-centered, whereas the remainder showed changes in receptive field location, boundaries, or responsiveness that rendered the response patterns hybrid or occasionally head-centered. These results suggest that FEF may act as a transitional step in an evolution of coordinates between LIP/MIP and SC. The persistence across cortical areas of mixed representations that do not provide unequivocal location labels in a consistent reference frame has implications for how these representations must be read out. NEW & NOTEWORTHY How we perceive the world as stable using mobile retinas is poorly understood. We compared the stability of visual receptive fields across different fixation positions in three visuomotor regions. Irregular changes in receptive field position were ubiquitous in intraparietal cortex, evident but less common in the frontal eye fields, and negligible in the superior colliculus (SC), where receptive fields shifted reliably across fixations. Only the SC provides a stable labeled-line code for stimuli across saccades.

Keywords: coordinate transformation; frontal eye field; intraparietal cortex; superior colliculus; visual saccade.

Fig. 1.

Schematic of the connections between the areas LIP/MIP, SC, and FEF, their visual inputs, and their projections to the brain stem saccade generator. The LIP/MIP and FEF are highly interconnected and send excitatory projections to the intermediate and deep layers of the SC (continuous arrows indicate direct projections). The FEF also sends inhibitory indirect projections to the SC through the caudate and the substantia nigra pars reticulata (dotted arrows indicate indirect projections). Both the SC and the FEF directly project to the various areas of the brain stem saccade generator system. The LIP/MIP and the FEF receive visual inputs from extrastriate visual areas. The SC receives visual inputs mainly in its superficial layer from the primary and secondary visual cortices and the FEF, and also directly from the retina (for reviews, see Blatt et al. 1990; Schall et al. 1995; Sparks and Hartwich-Young 1989). Connections between oculomotor areas are shown in gray and visual inputs in black.

Fig. 2.

Stimuli, task, and classification system of receptive fields. A: locations of stimuli and initial fixations. Varying the initial fixation permits the separation of eye- and head-centered reference frames by measuring the relative alignment of the responses in head- and eye-centered coordinates. B: task. Each trial starts with the appearance of a fixation light, which the monkey is required to fixate. A target then appears, but the monkey needs to wait until the fixation goes out before making a saccade to the target. C: predominantly eye-centered response patterns. The 3 tuning curves obtained for the 3 initial fixation locations align best in eye-centered coordinates (perfect alignment, R_eye ≈ 1; right), whereas in head-centered coordinates (left), they are shifted by the distance between the initial eye positions (i.e., steps of 12° in the present task, resulting in R_head < 1). C1 and C2 depict closed and open receptive fields, respectively. The classification metric is applicable in both cases. D: predominantly head-centered responses. The pattern is the opposite of that in C: the 3 tuning curves are aligned in head-centered coordinates and separated by 12° in eye-centered coordinates. D1 and D2 show that the classification is appropriate for both closed and open receptive fields. E: hybrid-partial shift response pattern. The 3 tuning curves are not well aligned in either head- or eye-centered coordinates: as the initial eye direction shifts left or right (red or blue, respectively), the tuning curves only partially move apart, by less than 12°. E1 and E2 show that both closed and open receptive fields are classified as hybrid-partial shift when the shift is less than the distance between initial fixations. F: hybrid-complex coordinates. The initial eye location affects the shape, gain, and/or alignment of the tuning curves in unpredictable ways that have no obvious relationship in either eye- or head-centered coordinates. G–J: schematics of population analysis. When R_head is plotted vs. R_eye, the data points should lie below the line of slope = 1 if the reference frame is predominantly eye-centered (G), above the line of slope = 1 if head-centered (H), along the line of slope = 1, but at positive values, if hybrid-partial shift (I), and randomly if hybrid-complex (J).

Fig. 3.

Examples of responses in the FEF. A–J: each panel shows the tuning curves for various example cells during the sensory or motor period (see materials and methods). The tuning curves are plotted in both head-centered (left) and eye-centered coordinates (right), and the reference frame indexes R_head and R_eye are indicated. A–D show examples of eye-centered responses (R_eye statistically higher than R_head) during the sensory period (A and C) and during the motor burst (B and D). The two responses in A and B (from the same cell at different time windows) show a complete sampling of the receptive field, whereas the examples in C and D show a partial sampling of the receptive fields. E: head-centered responses (R_eye statistically smaller than R_head) during the sensory period. F: head-centered responses during the motor burst. G: hybrid-complex responses (R_eye not statistically different from R_head and both not statistically different from zero) during the sensory period. H: hybrid-partial shift responses (R_eye not statistically different from R_head but at least one of them statistically higher than zero) during the motor burst. I: untuned responses during the sensory period. J: untuned responses during the motor burst. The reference frame index R was not calculated for the responses not significantly modulated by the target location as shown in I and J. In A–J, the thin, colored horizontal lines represent the average baselines for the 3 different fixation locations. Ipsi, ipsilateral; Contra, contralateral.

Fig. 4.

Reference frames in the FEF population. A and B: the reference frame indexes in head-centered and eye-centered coordinates are plotted for spatially selective cells in each time window: visual sensory (A) and visual motor (B). Responses are classified as eye-centered if the 95% confidence interval of eye-centered coefficient was positive, larger than, and nonoverlapping with the 95% confidence interval of head-centered coefficient (bootstrap analysis; see materials and methods); these responses are indicated in orange. Responses were classified as head-centered with the opposite pattern (blue). Finally, hybrid-partial shift reference frames (dark gray) have non-zero overlapping 95% confidence intervals, whereas hybrid-complex responses (light gray) have reference frame indexes not statistically different from zero. The pie charts summarize the proportion responses classified as eye-centered, head-centered, and the subtypes of hybrid for each time window. C and D: time course of the average eye-centered (black) and head-centered (gray) reference frame indexes for the FEF population of spatially selective responses (see materials and methods). The indexes were calculated in bins of 100 ms, sliding with a step of 50 ms and averaged across the population. Trials were aligned at target onset (C) and at saccade onset (D). Filled circles indicate that the difference between the 2 average indexes is statistically significant (t-test for each bin, P < 0.05). The time frames displayed in C and D overlap and include the variable time to the offset of the fixation light as well as saccade reaction time (averaging ~200 ms) as indicated by the shaded boxes.

Fig. 5.

Reference frame indexes during sensory and motor periods in LIP/MIP, FEF, and SC. A and B: the percentage of cells classified as eye-centered (orange), head-centered (blue), and hybrid (dark gray, hybrid-partial shift; light gray, hybrid-complex) are shown for all spatially selective cells in LIP/MIP, FEF, and SC in the sensory period (A) and in the motor period (B). For the sensory response, the time window is the 500 ms immediately after target onset. For the motor response, the time window changes with the saccade duration, ranging from −20 (SC), −50 (FEF), or −100 ms (LIP/MIP) from saccade onset to saccade offset (see materials and methods). For the FEF, the data are the same as in Fig. 4, A and B. C and D: time course of the eye-centered reference frame indexes for the populations of spatially selective cells (C) and for all recorded cells (D) in LIP/MIP (tan), FEF (green), and SC (dark red). Filled circles indicate that the eye-centered correlation coefficient was significantly larger than the head-centered one (2-tailed t-test for each bin, P < 0.05). Gray boxes indicate approximate timing of fixation light offset as well as saccade onset, as in Fig. 4, C and D. The FEF data are the same as in Fig. 4, C and D. The LIP/MIP and SC data were collected in studies by Lee and Groh (2012, and Mullette-Gillman et al. (2005, .