Flexible mechanisms underlie the evaluation of visual confidence

Abstract

Visual processing is fraught with uncertainty: The visual system must attempt to estimate physical properties despite missing information and noisy mechanisms. Sometimes high visual uncertainty translates into lack of confidence in our visual perception: We are aware of not seeing well. The mechanism by which we achieve this awareness--how we assess our own visual uncertainty--is unknown, but its investigation is critical to our understanding of visual decision mechanisms. The simplest possibility is that the visual system relies on cues to uncertainty, stimulus features usually associated with visual uncertainty, like blurriness. Probabilistic models of the brain suggest a more sophisticated mechanism, in which visual uncertainty is explicitly represented as probability distributions. In two separate experiments, observers performed a visual discrimination task in which confidence could be determined by the cues available (contrast and crowding or eccentricity and masking) or by their actual performance, the latter requiring a more sophisticated mechanism than cue monitoring. Results show that observers' confidence followed performance rather than cues, indicating that the mechanisms underlying the evaluation of visual confidence are relatively complex. This result supports probabilistic models, which imply the existence of sophisticated mechanisms for evaluating uncertainty.

Fig. 1.

Stimuli used in experiment 1. Stimuli consisted of five Gabor patches arranged on a cross. The central patch is the target, and the others are distractors. When viewed at high eccentricities, vertical distractors (B) induce a crowding effect on the target: The orientation of the target tends to assimilate with the similar orientation of the distractors, making small deviations from the vertical more difficult to discriminate. This effect does not occur with horizontal distractors (A), which are less similar in orientation. The crowding effect can be experienced by holding the figure at arms' length and fixating a few centimeters off the targets.

Fig. 2.

Crowding effect on performance. In the baseline condition observers chose between two stimuli with the same distractor orientation and the same target contrast (i.e., they were identical with the exception of target orientation, which was random). We varied the contrast of the stimulus and the orientation of the distractors across trials. (A) Data for observer AF. The blue and red circles represent measured frequency correct with horizontal and vertical distractors. Psychometric functions were fit to the data to summarize the effect of contrast on performance. (B) Psychometric functions for all six observers: Performance was systematically lower with vertical distractors than with horizontal ones (at equal contrast levels), showing that vertical distractors reliably induce a crowding effect.

Fig. 3.

Principle of the experiment. The red and blue curves describe hypothetical psychometric functions: They give an observer's expected performance as a function of contrast in the crowded and uncrowded conditions, as in Fig. 2. Suppose we always give the choice between a fixed, uncrowded stimulus with contrast x_U and another, crowded stimulus with contrast x_C. How can we set x_C to make the observer feel more confident about the crowded stimulus? According to the cue-monitoring hypothesis, two cues to uncertainty are available: contrast and crowdedness. If the observer picks crowdedness as a cue, then he or she will always prefer the uncrowded stimulus, no matter what the value of x_C. If the observer uses contrast as a cue, then he or she should prefer the crowded stimulus as soon as . Alternatively, if confidence follows performance, what the observer should do is choose the crowded stimulus as soon as it yields a higher expected performance: here, any point beyond . As shown above the plot, for any , the predictions of the performance-based hypothesis differ from those of either of the two cue-based models.

Fig. 4.

Individual results for the uncrowded vs. crowded condition in experiment 1. The observer had to choose between two stimuli, one crowded and the other uncrowded, with target contrasts x_C and x_U. We plot the probability of choosing the crowded stimulus as a response surface. The solid black contours are contours of the expected performance ratio: means that the observer had equal probability of making a correct orientation judgment by picking either stimulus. The expected performance ratio is computed from the results of an independent baseline condition (Fig. 2) and shown along with 10–90% bootstrap quantiles (dashed lines) (SI Materials and Methods). The green dashed line is the line : Note that the contrast levels for crowded stimuli were on average higher, because the case is relatively uninteresting in the context of this experiment (all theories predict that the observer should choose the uncrowded stimulus). NM performed under different feedback conditions than the other five observers (Materials and Methods).

Fig. 5.

Idealized theoretical predictions for observer DU. We plot a response surface, as in Fig. 4. The Inset corresponds to the range of contrasts we tested the observer on (the exact range differed between observers). (Left) Contrast heuristic. The observer chooses the crowded stimulus whenever it has higher contrast than the uncrowded one. (Center) Crowdedness heuristic. The observer systematically chooses the uncrowded stimulus. (Right) Performance-based strategy. The observer chooses the crowded stimulus only if it affords higher expected performance.

Fig. 6.

Stimuli used in experiment 2. In experiment 2 the observer chose between masked and unmasked stimuli that differed in eccentricity. The underlying psychophysical task used Landolt's C stimuli is that the gap in the circle could be facing either up or down. Because stimuli were displayed at a random eccentricity, they were preceded by a cue that indicated the location the stimulus would appear in. The stimulus was flashed briefly immediately after the cue. In the masked condition, the stimulus was followed directly by a noise mask. In the unmasked condition, a blank interval (called the interstimulus interval, ISI), was inserted between the stimulus and the mask.

Fig. 7.

Results for the mixed condition in experiment 2. The same general format as in Fig. 4 is followed. The proximity of a stimulus is defined as the opposite of eccentricity. Stimuli with proximity 1 stand next to the fixation cross, and stimuli with proximity 0 are as far from the fixation cross as possible on the monitor used (corresponding to 17.9° of eccentricity). Eccentricity plays the same role here as contrast does in experiment 1. Observers chose between masked and unmasked stimuli, and we plot the probability of choosing the masked one. Again, we would expect that if observers followed the strategy of always picking the stimulus with lower eccentricity, the green dashed line should separate the blue and red regions. The black line represents the line of equal expected performance, with bootstrap 10% and 90% quantiles shown as dashed lines.