Sample size re-estimation in paired comparative diagnostic accuracy studies with a binary response

Abstract

Background: The sample size required to power a study to a nominal level in a paired comparative diagnostic accuracy study, i.e. studies in which the diagnostic accuracy of two testing procedures is compared relative to a gold standard, depends on the conditional dependence between the two tests - the lower the dependence the greater the sample size required. A priori, we usually do not know the dependence between the two tests and thus cannot determine the exact sample size required. One option is to use the implied sample size for the maximal negative dependence, giving the largest possible sample size. However, this is potentially wasteful of resources and unnecessarily burdensome on study participants as the study is likely to be overpowered. A more accurate estimate of the sample size can be determined at a planned interim analysis point where the sample size is re-estimated.

Methods: This paper discusses a sample size estimation and re-estimation method based on the maximum likelihood estimates, under an implied multinomial model, of the observed values of conditional dependence between the two tests and, if required, prevalence, at a planned interim. The method is illustrated by comparing the accuracy of two procedures for the detection of pancreatic cancer, one procedure using the standard battery of tests, and the other using the standard battery with the addition of a PET/CT scan all relative to the gold standard of a cell biopsy. Simulation of the proposed method illustrates its robustness under various conditions.

Results: The results show that the type I error rate of the overall experiment is stable using our suggested method and that the type II error rate is close to or above nominal. Furthermore, the instances in which the type II error rate is above nominal are in the situations where the lowest sample size is required, meaning a lower impact on the actual number of participants recruited.

Conclusion: We recommend multinomial model maximum likelihood estimation of the conditional dependence between paired diagnostic accuracy tests at an interim to reduce the number of participants required to power the study to at least the nominal level.

Trial registration: ISRCTN ISRCTN73852054 . Registered 9th of January 2015. Retrospectively registered.

Keywords: Diagnostic accuracy; Interim analysis; Sample-size re-estimation; Sensitivity; Specificity; Study design.

Fig. 1

Simulated power of sample size specified by the true TPPR in equation 1 when TPR_A=0.9, TPR_B=0.81 and π=0.45

Fig. 2

Simulated power of re-estimation method across various interim sample sizes and levels of true TPPR when TPR_A = 0.9, TPR_B = 0.81 and π = 0.45