Confidence in Context: Perceived Accuracy of Quantitative Estimates Decreases With Repeated Trials

Abstract

Across seven studies (combined N = 5,484), we demonstrated that confidence in one's judgments decreases over a series of quantitative estimates. This finding was robust to various methods of confidence elicitation, the presence of incentives, and different estimation topics (Studies 1, 2, and 4). Our results also stand in contrast to participant expectations (Study 3). The phenomenon does not appear to be driven by fatigue, lack of effort, or various explanations based on incorporating uncertainty from prior judgments into subsequent ones. Our findings suggest that rather than evaluating confidence in isolation, participants evaluate confidence in reference to their stated confidence on earlier judgments. We theorize that confidence in earlier judgments increases in hindsight because of biased forgetting of disconfirming evidence. As a result, confidence in subsequent judgments appears to be comparatively lower (preregistered Studies 5-7). We discuss the implications for confidence research and consumer, organizational, and policy decision-making.

Keywords: confidence; judgment; metacognition; open data; open materials; overconfidence; overprecision; postdecision; preregistered.