Foveal analysis and peripheral selection during active visual sampling

Abstract

Human vision is an active process in which information is sampled during brief periods of stable fixation in between gaze shifts. Foveal analysis serves to identify the currently fixated object and has to be coordinated with a peripheral selection process of the next fixation location. Models of visual search and scene perception typically focus on the latter, without considering foveal processing requirements. We developed a dual-task noise classification technique that enables identification of the information uptake for foveal analysis and peripheral selection within a single fixation. Human observers had to use foveal vision to extract visual feature information (orientation) from different locations for a psychophysical comparison. The selection of to-be-fixated locations was guided by a different feature (luminance contrast). We inserted noise in both visual features and identified the uptake of information by looking at correlations between the noise at different points in time and behavior. Our data show that foveal analysis and peripheral selection proceeded completely in parallel. Peripheral processing stopped some time before the onset of an eye movement, but foveal analysis continued during this period. Variations in the difficulty of foveal processing did not influence the uptake of peripheral information and the efficacy of peripheral selection, suggesting that foveal analysis and peripheral selection operated independently. These results provide important theoretical constraints on how to model target selection in conjunction with foveal object identification: in parallel and independently.

Keywords: attention; classification image; eye movement control; perceptual decision-making.

Fig. 1.

Hypothetical temporal weighting functions for foveal analysis and peripheral selection. (A) Strict serial model: peripheral information is analyzed only once foveal processing is complete. (B) A weaker version of the serial model in which peripheral information is processed once some criterion amount of foveal analysis is complete. (C) Parallel model in which foveal analysis and peripheral selection start together. In A–C, the time window for peripheral selection is shorter than that for foveal analysis, reflecting the primary importance of the latter. (D) Manipulation of foveal load. As foveal processing difficulty is increased, more time is taken to analyze the foveal information. The time window for peripheral selection extends as well, but by a smaller amount. In addition, the gain of peripheral processing is lower, resulting in attenuation of the amplitude of the weighting function.

Fig. 2.

Illustration of the paradigm. Trials start with a preview of variable duration. The preview is replaced by a sequence of test images, with each image shown for two video frames (∼24 ms per image), for a total duration of ∼750 ms, or 32 images at an 85-Hz monitor refresh rate. The peripheral target is signaled by its higher average luminance contrast (straight up in this figure). The mean target and nontarget contrasts are equidistant from the preview contrast. The fixation pattern remains at the preview contrast. The contrast of all patterns is perturbed independently with zero-mean Gaussian noise (σ = 0.15). The fixation pattern can be tilted clockwise or anticlockwise from vertical (clockwise in this example). The target pattern can be tilted in the same or a different direction (anticlockwise in this example). The direction of tilt of the remaining patterns is determined randomly and independently, so that their tilt conveys no information about the likely target orientation (i.e., the orientations of the peripheral patterns are uncorrelated). The orientation of all four patterns is also perturbed independently with zero-mean Gaussian noise (σ = 6°). The mean pattern tilt was either 1° or 2° (as in this figure).

Fig. 3.

Raw temporal classification images for foveal identification (orientation noise) (A and B) and peripheral selection (C and D) for one observer. Note that display onset refers to the start of the noisy test image sequence; it is not the same as stimulus onset, due to the preview during which all patterns were already present (Fig. 2). The three triangles in the display-aligned panels (A and C) correspond to the 25th, 50th (median), and 75th percentiles of the fixation duration distribution. The gray-shaded box in the saccade-aligned plots (B and D) corresponds to the mean saccade duration for this observer.

Fig. 4.

Noise classification accuracy for foveal identification and peripheral selection, averaged across eight observers. In the saccade-aligned panel (Right), the average movement duration is shown by the vertical shaded box. The shaded region around the functions corresponds to the 95% confidence interval across the subject pool. Note that in the saccade-aligned plot, fewer trials contribute to the extreme time points (i.e., long before movement onset and long after movement offset). To align the noise samples on movement onset, we assigned the sample during which the movement started to time 0. This relatively crude alignment means that the “true” onset of the saccade relative to the start of the noise sample is accurate within the duration of an individual noise frame (i.e., ∼24 ms). Given the large amount of data collected for each observer, the average starting point of the movement will lie near the midpoint of the noise frame.

Fig. 5.

Noise classification accuracy for foveal identification (A and B) and peripheral selection (C and D), averaged across eight observers. Each panel contains a separate function for the two levels of foveal processing load. In the saccade-aligned plots (B and D), the saccade duration is shown by the shaded vertical box. The shaded regions around the functions show the 95% confidence intervals.

Fig. 6.

Comparison of foveal tilt discrimination under single- and dual-task conditions. (A) Single-task performance. Observers viewed a single pattern at fixation, which fluctuated in orientation and contrast in exactly the same way as the foveal target in the main experiment. After a variable interval, the pattern was extinguished, and the observer generated a vertical, upward saccade to a Gaussian noise-patch. Accuracy is averaged across observers; the error bars are within-subject SEMs. (B) Dual-task tilt performance (i.e., with a concurrent peripheral selection demand) as a function of single-task performance. Single-task accuracy was found by plugging the mean first fixation duration into the functions that relate tilt discrimination accuracy to viewing time (as in A).