The impact of evidence reliability on sensitivity and bias in decision confidence

Abstract

Human observers effortlessly and accurately judge their probability of being correct in their decisions, suggesting that metacognitive evaluation is an integral part of decision making. It remains a challenge for most models of confidence, however, to explain how metacognitive judgments are formed and which internal signals influence them. While the decision-making literature has suggested that confidence is based on privileged access to the evidence that gives rise to the decision itself, other lines of research on confidence have commonly taken the view of a multicue model of confidence. The present study aims at manipulating one such cue: the perceived reliability of evidence supporting an initial decision. Participants made a categorical judgment of the average color of an array of eight colored shapes, for which we critically manipulated both the distance of the mean color from the category boundary (evidence strength) and the variability of colors across the eight shapes (evidence reliability). Our results indicate that evidence reliability has a stronger impact on confidence than evidence strength. Specifically, we found that evidence reliability affects metacognitive readout, the mapping from subjectively experienced certainty to expressed confidence, allowing participants to adequately adjust their confidence ratings to match changes in objective task performance across conditions. (PsycINFO Database Record

Figure 1

(A) Sample stimuli, showing the four difficulty conditions in the 2 (mean) × 2 (variance) design. Color values are made more extreme for illustrative purposes. (B) Design of the color task; participants had to indicate whether an array of eight colored shapes was on average more red or more blue by pressing the left or right response key. After making their response, the confidence scale was presented on screen and participants were given unlimited time to choose how confident they were that their last response was correct. RSI: response-stimulus interval; max. = maximum.

Figure 2

(A) Distributions of confidence levels displayed for correct and error trials separately. (B) Confidence for the four difficulty color conditions (together for both error and correct trials). All error bars are within-subject confidence intervals (Loftus & Masson, 1994).

Figure 3

(A) Standardized regression coefficients for three different models to predict confidence. Colored xs reflect the respective parameters that were not considered in the regression model. (B) Signed t values for three different models to predict confidence. The horizontal dashed lines highlight the critical t values, so that model parameters above or below the dashed, horizontal lines are significantly different from zero. (C) Mean R² and (D) mean BICs for these models. RT = reaction time; BIC = Bayesian information criterion.

Figure 4

Nonparametric Type II SDT parameter estimates as a function of difficulty. (A) A_ROC expresses metacognitive sensitivity. (B) B_ROC expresses metacognitive bias. All error bars are within-subject confidence intervals (Loftus & Masson, 1994). SDT = signal detection theory; ROC = receiver operating characteristic.