Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy

Abstract

Purpose: To identify radiologists' characteristics associated with interpretive performance in screening mammography.

Materials and methods: The study was approved by institutional review boards of University of Washington (Seattle, Wash) and institutions at seven Breast Cancer Surveillance Consortium sites, informed consent was obtained, and procedures were HIPAA compliant. Radiologists who interpreted mammograms in seven U.S. regions completed a self-administered mailed survey; information on demographics, practice type, and experience in and perceptions of general radiology and breast imaging was collected. Survey data were linked to data on screening mammograms the radiologists interpreted between January 1, 1998, and December 31, 2005, and included patient risk factors, Breast Imaging Reporting and Data System assessment, and follow-up breast cancer data. The survey was returned by 71% (257 of 364) of radiologists; in 56% (205 of 364) of the eligible radiologists, complete data on screening mammograms during the study period were provided; these data were used in the final analysis. An evaluation of whether the radiologists' characteristics were associated with recall rate, false-positive rate, sensitivity, or positive predictive value of recall (PPV(1)) of the screening examinations was performed with logistic regression models that were adjusted for patients' characteristics and radiologist-specific random effects.

Results: Study radiologists interpreted 1 036 155 screening mammograms; 4961 breast cancers were detected. Median percentages and interquartile ranges, respectively, were as follows: recall rate, 9.3% and 6.3%-13.2%; false-positive rate, 8.9% and 5.9%-12.8%; sensitivity, 83.8% and 74.5%-92.3%; and PPV(1), 4.0% and 2.6%-5.9%. Wide variability in sensitivity was noted, even among radiologists with similar false-positive rates. In adjusted regression models, female radiologists or fellowship-trained radiologists had significantly higher recall and false-positive rates (P < .05, all). Fellowship training in breast imaging was the only characteristic significantly associated with improved sensitivity (odds ratio, 2.32; 95% confidence interval: 1.42, 3.80; P < .001) and the overall accuracy parameter (odds ratio, 1.61; 95% confidence interval: 1.05, 2.45; P = .028).

Conclusion: Fellowship training in breast imaging may lead to improved cancer detection, but it is associated with higher false-positive rates.

Figure 1:

Performance of 187 U.S. radiologists who interpreted images from screening mammographic examinations (with images from one or more examinations associated with a cancer diagnosis). Sensitivity and false-positive rate are shown for each radiologist. Specificity was calculated by subtracting false-positive rate from one. Size of circle represents number of screening mammograms interpreted by that radiologist that were associated with a cancer diagnosis, with larger circles representing more cancers. Small circle may represent a radiologist with screening mammograms associated with one or two cancers. Normalized partial area under the curve is 0.82. Two areas are in color to highlight radiologists with both sensitivity and false-positive rate in highest (≥75th percentile in green) and lowest (≤25th percentile in red) rating for interpretive performance on basis of national BCSC benchmarks for screening performance (11).

Figure 2:

Unadjusted recall rate and PPV₁ for 203 U.S. radiologists with respect to theoretic cancer detection rates per 1000 screening mammographic examinations. Red circles designate fellowship-trained radiologists and blue circles designate non–fellowship-trained radiologists. Four curved lines represent theoretical cancer detection rates for a given PPV₁ and recall rate. Cancer detection rate is defined as the number of true-positive mammograms for every 1000 screening examinations. For example, a radiologist with PPV₁ of five cancers per 100 positive screening examinations (0.05) and a recall rate of five screening examinations with positive results per 100 screening examinations (0.05) would have a cancer detection rate of 2.5 per 1000 (0.05·0.05 = 0.0025) screening examinations. This rate would be between the lines that represent cancer detection rates of two per 1000 screening examinations and four per 1000 screening examinations, the range for most of the radiologists in this study.