Evaluation of bias in diagnostic-test sensitivity and specificity estimates computed by discrepant analysis

Abstract

When a new diagnostic test is potentially more sensitive than the reference test used to classify persons as infected or uninfected, a substantial number of specimens from infected persons may be reference-test negative but new-test positive. Discrepant analysis involves the performance of one or more additional tests with these specimens, reclassification as infected those persons for whom the new-test-positive results are confirmed, and recalculation of the estimates of new-test sensitivity and specificity by using the revised classification. This approach has been criticized because of the bias introduced by the selective use of confirmation testing. Under conditions appropriate for evaluating a nucleic acid amplification (NAA) test for Chlamydia trachomatis infection with cell culture as the reference test, we compared the bias in estimates based on the discrepant-analysis classification of persons as infected or uninfected with that in estimates based on the culture classification. We concluded that the bias in estimates of NAA-test specificity based on discrepant analysis is small and generally less than that in estimates based on culture. However, the accuracy of discrepant-analysis-based estimates of NAA-test sensitivity depends critically on whether culture specificity is equal to or is slightly less than 100%, and it is affected by competing biases that are not fully taken into account by discrepant analysis.

FIG. 1

Bias in culture-based (——) and discrepant-analysis-based (–––) estimates of LCR specificity (culture specificity = 100%; LCR sensitivity = 85%; LCR sensitivity is the same for culture-positive and culture-negative specimens).

FIG. 2

Bias in culture-based (——) and discrepant-analysis-based (–––) estimates of LCR sensitivity (prevalence of infection = 5%; LCR specificity = 95%). Culture/LCR independent refers to cases in which LCR sensitivity is the same for culture-positive and culture-negative specimens. Culture/LCR dependent refers to cases in which LCR sensitivity is moderately higher for culture-positive than for culture-negative specimens.

FIG. 3

Comparison of bias in culture-based and discrepant-analysis-based estimates of LCR sensitivity (culture specificity <100%; LCR sensitivity is moderately higher for culture-positive than for culture-negative specimens). The shaded areas indicate combinations of test performance characteristics and prevalence of infection for which the discrepant-analysis-based estimate is less biased than the culture-based estimate.