Assessing diagnostic tests once an optimal cutoff point has been selected

Abstract

The specificity and sensitivity of a quantitative diagnostic test depends on the chosen cutoff point. The common practice of selecting a cutoff point that maximizes the specificity plus the sensitivity, as judged from the observed test results, is studied here by simulation. Test performance is on average assessed too optimistically by this procedure--a phenomenon of importance when sample sizes are small. For example, the average positive bias is up to 15% of the test performance for sample sizes of 25. Furthermore, binomial calculated standard errors of specificity and sensitivity estimates are incorrect. A Monte Carlo statistical method--the "bootstrap procedure"--is applied to correct for bias and to estimate standard errors, including the standard error of the optimal cutoff point. Independent and paired comparisons of two diagnostic tests are also considered when optimal cutoff points have been selected. For this purpose, binomial statistical tests behave satisfactorily. Examples of power functions are presented.