Continued misinterpretation of confidence intervals: response to Miller and Ulrich

Abstract

Miller and Ulrich (2015) critique our claim (Hoekstra et al., Psychonomic Bulletin & Review, 21(5), 1157-1164, 2014), based on a survey given to researchers and students, of widespread misunderstanding of confidence intervals (CIs). They suggest that survey respondents may have interpreted the statements in the survey that we deemed incorrect in an idiosyncratic, but correct, way, thus calling into question the conclusion that the results indicate that respondents could not properly interpret CIs. Their alternative interpretations, while correct, cannot be deemed acceptable renderings of the questions in the survey due to the well-known reference class problem. Moreover, there is no support in the data for their contention that participants may have had their alternative interpretations in mind. Finally, their alternative interpretations are merely trivial restatements of the definition of a confidence interval, and have no implications for the location of a parameter.

Keywords: Bayesian statistics; Statistical inference; Statistics.

Fig. 1

Probability that a 95 % Student’s t ₉ CI contains the true value as a function of the ratio of the sample standard deviation s to the true standard deviation σ

Fig. 2

Top: Proportion of sample endorsing each statement as a function of self-rated expertise. Each series 1–6 represents the corresponding statement. If expertise is NA, then the respondent did not respond to this question. The vast majority of missing expertise ratings is from first-year students (86 %) or Master students (9 %). Bottom: The distribution of self-rated statistical expertise in the sample