Eye movement statistics in humans are consistent with an optimal search strategy

Abstract

Most models of visual search are based on the intuition that humans choose fixation locations containing features that best match the features of the target. The optimal version of this feature-based strategy is what we term "maximum a posteriori (MAP) search." Alternatively, humans could choose fixations that maximize information gained about the target's location. We term this information-based strategy "ideal search." Here we compare eye movements of human, MAP, and ideal searchers in tasks where known targets are embedded at unknown locations within random backgrounds having the spectral characteristics of natural scenes. We find that both human and ideal searchers preferentially fixate locations in a donut-shaped region around the center of the circular search area, with a high density of fixations at top and bottom, while MAP searchers distribute their fixations more uniformly, with low density at top and bottom. Our results argue for a sophisticated search mechanism that maximizes the information collected across fixations.

Figure 1

Visual search in a probabilistic framework. The searcher starts with some initial prior beliefs (represented as probabilities) about the target being located at every potential location in the visual scene. During the first fixation, the searcher encodes visual data from all the potential target locations and uses it to update its prior beliefs to posterior beliefs. If the posterior probability at some location exceeds a criterion, then the search is stopped and the location with the largest posterior probability is picked as the target location. If the criterion is not exceeded, the searcher chooses the next location to fixate, and the process repeats until the searcher finds the target. The searcher's representation of its own visual limitations (i.e., the detectability of the target across the visual field) can affect how it updates its beliefs and how it chooses the next fixation location. The searcher's memory limitations can affect the beliefs stored across fixations. An ideal searcher that aims to find the target quickly (i) has perfect memory, (ii) has precise knowledge about the target, about the statistical properties of the visual scene in which the target is embedded, and about its own visual system, and (iii) makes the eye movements that that on average will gain the most information about the target's location.

Figure 2

Target visibility maps and search performance. (A) To measure visibility maps, psychometric functions (detection accuracy as a function of target contrast) were measured at the twenty-five locations indicated by the circles (the circles did not appear in the display). (B) Proportion of correct responses in the fovea, as a function of target rms contrast, for four levels of background noise rms contrast (from Najemnik & Geisler, 2005). (C) Proportion of correct responses at 4.5 deg eccentricity, as a function of target contrast, for the same four levels of background contrast. (D) Detection threshold contrast power (the square of rms contrast) of the target as a function of noise contrast power in the fovea and at 4.5 degrees eccentricity, for two observers. The detection threshold is defined to be the contrast power of the target corresponding to 82% correct. (E) Contour plot of the visibility map (d′ as a function of retinal position) for one combination of target and background contrast, based on the average psychometric-function data from the two observers. Note that the signal-to-noise ratio d′ is monotonically related to the probability of a correct response in the detection task (see Methods). (F) Median number of fixations on correct search trials, as a function of target detectability at the center of the fovea, for two human searchers (symbols), the ideal searcher (solid curves) and the MAP searcher (dashed curves). (The data points and solid curves are from Najemnik & Geisler, 2005.) The histograms show the error rates for the human searchers (gray) and the ideal searcher (white)

Figure 3

Average spatial distribution of fixation locations across the search area for ideal, map, and human searchers. The color temperature indicates the relative proportion of fixations at each display location. The dashed circles indicate the display region containing the 1/f noise texture. Each of the plots in A through E is based upon approximately 17,000 fixations. The smooth distributions were obtained using a Parzen window with a standard deviation of 0.2 deg.

Figure 4

Direction histograms of fixation location relative to the center of the display. The histograms were obtained with a sliding summation window having a width of 45 deg. All three histograms were normalized by the maximum frequency for the human searchers.

Figure 5

Distribution of saccade vectors for human, ideal, and MAP searchers. In each plot, the saccade take-off point is taken to be the origin, and the color temperature reflects the density of the landing points relative to the origin. In these plots, a rightward saccade has a direction of 0 deg and an upward saccade a direction of 90 deg. The color temperature scale is the same as in Figure 3. The white contours show the relative proportion of saccades in each direction (axis on the right).

Figure 6

Average fixation distance from the center of the search display for human, ideal, and MAP searchers. (A) Average distance from the center of the display as a function of search task difficulty, as indexed by the average number of fixations to find the target. The data (open symbols) and ideal predictions (solid circles) are from Geisler et al. (2006). (B) Average fixation distance from the center of the display as a function of the number of saccades elapsed within the search trial in the present study (averaged across task difficulty).