Accuracy of screening mammography using single versus independent double interpretation

Abstract

Objective: We conducted an analysis among 31 community radiologists to identify the average change in screening mammography interpretive accuracy afforded by independent double interpretation.

Materials and methods: We assessed interpretive accuracy using a stratified random sample of test mammograms that included 30 women with cancer and 83 without. Radiologists were unaware of clinical information and of each other's assessments. We describe accuracy for individual radiologists and for double interpretation, including average sensitivity, specificity, diagnostic likelihood ratios positive and negative, and area under the receiver operating characteristic (ROC) curve. We also assessed weighted and nonweighted kappa statistics among all 465 pairs of radiologists and 31,465 pairs of unique pairs. The assessment for double interpretations used the "highest" (i.e., most abnormal) assessment of the two radiologists. We calculated the difference between each radiologist's individual accuracy and the average accuracy across that radiologist's 30 double interpretations.

Results: We found the following average accuracy statistics for individual radiologists: sensitivity, 79%; specificity, 81%; diagnostic likelihood ratio positive, 5.53; diagnostic likelihood ratio negative, 0.26; and area under the ROC curve, 0.85. The mean kappa statistic among radiologists for cancer cases increased with double interpretation from 0.59 to 0.70, and for noncancer cases from 0.30 to 0.34. Double interpretation resulted in an average increase in sensitivity of 7%, an average decrease in specificity of 11%, a decrease in diagnostic likelihood ratio positive of 2.35, a decrease in diagnostic likelihood ratio negative of 0.06, and an increase in area under the ROC curve of 0.02.

Conclusion: Independent double interpretation does not increase accuracy as measured by the area under the ROC curve.