Predictive activity in macaque frontal eye field neurons during natural scene searching

Abstract

Generating sequences of multiple saccadic eye movements allows us to search our environment quickly and efficiently. Although the frontal eye field cortex (FEF) has been linked to target selection and making saccades, little is known about its role in the control and performance of the sequences of saccades made during self-guided visual search. We recorded from FEF cells while monkeys searched for a target embedded in natural scenes and examined the degree to which cells with visual and visuo-movement activity showed evidence of target selection for future saccades. We found that for about half of these cells, activity during the fixation period between saccades predicted the next saccade in a sequence at an early time that precluded selection based on current visual input to a cell's response field. In addition to predicting the next saccade, activity during the fixation prior to two successive saccades also predicted the direction and goal of the second saccade in the sequence. We refer to this as advanced predictive activity. Unlike activity indicating the upcoming saccade, advanced predictive activity occurred later in the fixation period, mirroring the order of the saccade sequence itself. The remaining cells without advanced predictive activity did not predict future saccades but reintroduced the signal for the upcoming saccade at an intermediate time in the fixation period. Together these findings suggest that during natural visual search the timing of FEF cell activity is consistent with a role in specifying targets for one or more future saccades in a search sequence.

Fig. 1.

Scene search task. A: sample scene with embedded target fly. Monkey's eye traces during the trial appear in yellow. Bottom right: zoom in on target for better visibility. B: extrsaction process for saccades of this trial. Blue, response-field; red, anti-response-field; green, neutral fields; gray, excluded border zones. C: polar plot of vector endpoints for all saccades made while recording activity from the neuron with response field depicted in B.

Fig. 2.

Saccades excluded from analysis. Activity recorded at the colored fixation spots were excluded for varying reasons. A: although saccade A is toward the response field, activity recorded while fixating at the gray spot is excluded because the vector of the preceding saccade also was directed toward the response field. B: saccade B was excluded in our 2nd analysis because due to the size of frontal eye field (FEF) response fields, the portion of the scene located around the gray spot was in the cell's response field for 2 successive fixation periods (blue and green spots). Therefore early increases in activity while fixating at the green spot could have been due to prior activation during the previous fixation period. C: 2nd goal activity recorded while fixating at the location marked by the blue spot was excluded because although the 2nd goal (the endpoint of vector C) was toward the response field, the 2nd saccade (saccade B) was as well.

Fig. 3.

Early prediction times during the scene search task. A: representative visual cell. Rows 1 and 2 show spike rasters and spike density curves for mean firing rates during the fixation period prior to saccades made into the response field (blue), and into the anti response field (red). Row 3 compares the firing rates between the 2 conditions and indicates the receiver operator characteristic (ROC) prediction time (green line). Black vertical lines indicate the beginning of the fixation period before the saccade (red dots in rasters). Activity occurring after the mean saccade latency shaded in gray. Row 4 displays the same cell's activity during the memory-guided saccade task. Activity is aligned by target onset (left) and saccade onset (right). Onset of visual response indicated by vertical green line. The cell fires strongly after target onset but not before the saccade. B: representative visuomovement cell. C: comparison between the mean visual response latency and the ROC prediction times for visual and visuomovement cell. Mean visual response latency was significantly greater than the ROC prediction times for either type of FEF cell.

Fig. 4.

Second saccade and second goal determinations. Solid arrows indicate examples of 2 successive saccades. Often a single trial yielded multiple saccade pairs for analysis. Activity obtained during the fixation period preceding the saccade pairs was analyzed for predictive activity (gray circle). A: 2nd saccade analysis. Instances in which the 2nd saccade was directed into the response field (top, solid blue arrow) were compared with cases in which the 2nd saccade was directed away from the response field (bottom, solid red arrow). These sequences were included because vectors of both the 1st saccade (solid green arrows) and the 2nd goal (dotted green arrows) fell in neutral areas far from the response field or its opposite direction. B: 2nd goal analysis. Instances in which the 2nd goal was within the response field (top, dotted blue arrow) were compared with cases in which the 2nd goal was located in a direction opposite to that of the response field (bottom, dotted red arrow). These sequences were included because vectors of both the 1st saccade and 2nd saccades (solid green arrows) fell in neutral areas far from the response field or its opposite. The circular inset in the lower right corner depicts the cell's response field (RF) in a manner identical to Fig. 1B.

Fig. 5.

Types of 2nd saccade and 2nd goal predictive activity. We found cells that displayed 2nd saccade and/or ^2nd goal predictive activity as well as cells that did neither. Left. firing rates during fixation periods in which the 2nd saccade was directed toward (blue) and away from (red) the response-field. Right. firing rates during fixation periods in which the 2nd goal was located either within (blue) or at a location opposite to the response-field (red). Black vertical line indicates the beginning of the fixation period preceding the pair of saccades. Vertical green line mark the time at which ROC analysis indicated that advanced predictive activity occurred. Time after the mean latency of the 1st saccade shaded in gray. Row 1: a cell that could predict only the 2nd saccade of a sequence but not the goal. Row 2: a cell that could only predict the 2nd goal of a sequence but not the 2nd saccade. Row 3: a cell that could predict both the 2nd saccade and the 2nd goal of a sequential pair of saccades. Row 4: a cell that did not display any advanced predictive activity.

Fig. 6.

Timing of predictive activity. A: comparison between prediction times for upcoming saccades and future saccades. Activity that predicts the upcoming (1st) saccade occurs significantly earlier in the fixation period than that of the 2nd goal or 2nd saccade (*). B: prediction times before the upcoming saccade. When FEF cells are divided into those with advanced predictive activity and those without, a clear distinction can be seen. FEF cells that combined both types of advanced predictive activity indicated the direction of the upcoming saccade significantly earlier than FEF cells that did not display advanced predictive activity (*). Cells with only 2nd goal activity also showed earlier prediction times, but this did not reach significance.

Fig. 7.

Shorter saccade latencies to target. Saccade latencies toward the target are plotted against saccade latencies toward other portions of the scene during the search. Each black dot represents mean latency data from 1 recording session. Regression line in solid black. Dotted line indicates expected values if there were no differences in latency between the 2 conditions.

Fig. 8.

Latency and amplitude for saccades to scene and target. A: comparison of latency of saccades to the target vs. saccades to nontarget portions of the image as a function of saccade order in the trial. *, number of saccade in trial where latency of saccade to target was significantly less than saccade to a nontarget part of the scene. B: comparison of saccade latencies and amplitudes for target and nontarget saccades. *, saccade amplitudes where latency of saccades to nontarget parts of scene were significantly longer than saccades to the target. Vertical bar indicates SE.

Fig. 9.

Relative timing of FEF visual and visuomovement cell activity for the generation of a sequence of 2 saccades. After the start of fixation, at relative time point A, cells with advanced predictive activity are the first to signal the direction of the upcoming saccade (S1). Later at time point B, cells without advanced predictive activity also signal the direction for S1. Later in the fixation period at time point C, advanced predictive cells signal the spatial goal (G2) and saccade vector (S2) for the eye movement that will follow the upcoming saccade. The relative times of these activities are based on the values illustrated in Fig. 6 and discussed in the text.

Fig. 10.

Distribution of FEF cells with and without advanced predictive activity. This diagram shows the relative numbers of visual neurons with no motor activity and visuomovement neurons with motor activity. The distribution of these 2 cell types across the groupings of cells with and without advanced predictive activity was roughly the same.