Bayesian sample size determination for diagnostic accuracy studies

Abstract

The development of a new diagnostic test ideally follows a sequence of stages which, among other aims, evaluate technical performance. This includes an analytical validity study, a diagnostic accuracy study, and an interventional clinical utility study. In this article, we propose a novel Bayesian approach to sample size determination for the diagnostic accuracy study, which takes advantage of information available from the analytical validity stage. We utilize assurance to calculate the required sample size based on the target width of a posterior probability interval and can choose to use or disregard the data from the analytical validity study when subsequently inferring measures of test accuracy. Sensitivity analyses are performed to assess the robustness of the proposed sample size to the choice of prior, and prior-data conflict is evaluated by comparing the data to the prior predictive distributions. We illustrate the proposed approach using a motivating real-life application involving a diagnostic test for ventilator associated pneumonia. Finally, we compare the properties of the approach against commonly used alternatives. The results show that, when suitable prior information is available, the assurance-based approach can reduce the required sample size when compared to alternative approaches.

Keywords: Bayesian assurance; binomial intervals; contingency tables; power calculations; sensitivity; specificity.

FIGURE 1

Left: The prior distributions for the sensitivity (red) and the prevalence (black) for the biomarker selection study (dashed lines) and the diagnostic accuracy study (solid lines). Right: The assurance curve showing the assurance achieved at different sample sizes for the diagnostic accuracy study

FIGURE 2

The prior predictive distributions of the number of patients with VAP (left) and the number of VAP patients who test positive (right) together with the observations (red)

FIGURE 3

A comparison of the sample sizes required based on power calculations (dashed) using a Wald interval (dark blue), Clopper‐Pearson (red), Agresti‐Coull (green), assurance (black, solid), and assurance based on non‐informative analysis priors (light blue). In each plot, there are three black curves relating to prior sample sizes of (from top to bottom) 25, 50, and 75

FIGURE 4

The width of 95% confidence or posterior probability intervals based on 100 simulations for the Wald interval (Wald), Clopper‐Pearson (CP), Agresti‐Coull (AC), Assurance (BAM), and Assurance using a non‐informative analysis prior (Non‐inf). The power/assurance used to choose the sample size was 0.5 (left) and 0.8 (right). The horizontal line is at the desired width of $w^{\ast }=0.18$