Receiver operating characteristic analysis for the detection of simulated microcalcifications on mammograms using hardcopy images

Abstract

The aim of this study was to compare mammography systems based on three different detectors--a conventional screen-film (SF) combination, an a-Si/CsI flat-panel (FP)-based detector, and a charge-coupled device (CCD)-based x-ray phosphor-based detector--for their performance in detecting simulated microcalcifications (MCs). 112-150 microm calcium carbonate grains were used to simulate MCs and were overlapped with a slab phantom of simulated 50% adipose/50% glandular breast tissue-equivalent material referred to as the uniform background. For the tissue structure background, 200-250 microm calcium carbonate grains were used and overlapped with an anthropomorphic breast phantom. All MC phantom images were acquired with and without magnification (1.8 X). The hardcopy images were reviewed by five mammographers. A five-point confidence level rating was used to score each detection task. Receiver operating characteristic (ROC) analysis was performed, and the areas under the ROC curves (A(z)s) were used to compare the performances of the three mammography systems under various conditions. The results showed that, with a uniform background and contact images, the FP-based system performed significantly better than the SF and the CCD-based systems. For magnified images with a uniform background, the SF and the FP-based systems performed equally well and significantly better than the CCD-based system. With tissue structure background and contact images, the SF system performed significantly better than the FP and the CCD-based systems. With magnified images and a tissue structure background, the SF and the CCD-based systems performed equally well and significantly better than the FP-based system. In the detection of MCs in the fibroglandular and the heterogeneously dense regions, no significant differences were found except that the SF system performed significantly better than the CCD-based system in the fibroglandular regions for the contact images.

Figure 1.

(a) Simulated calcifications were overlapped with the uniform background (top) and tissue structure background (bottom) and acquired with the FP-based mammography system with contact mode. The microcalcification size was 140–150 and 224–250 μm for the uniform and tissue structure backgrounds, respectively. The matrix size was 200 × 200 for all images. Six MCs were arranged in a 1 cm line shape (left) or 4 mm wide hexagonal shape (right). (b) Simulated calcifications were overlapped with the uniform background (top) and tissue structure background (bottom) and acquired with the FP-based mammography system with contact mode. The microcalcification size was 140–150 and 224–250 μm for the uniform and tissue structure backgrounds, respectively. The matrix size was 200 × 200 for all images. One MC was missing (square used as a control) for a 1 cm line shape (left) or 4 mm hexagonal shape (right). (c) Simulated MC clusters were overlapped with the uniform background (left) and tissue structure background (right) and acquired with the FP-based mammography system with contact mode. The MC sizes were 112–150 and 200–250 μm for the uniform and tissue structure backgrounds, respectively. Three MC inserts were randomly arranged in any three of four quadrants and an empty film inserted in the last quadrant, providing six controls.

Figure 2.

The diagram shows the template used to position and sandwich the MC phantom overlapped with a 5 cm 50%/50% slab phantom or an anthropomorphic breast phantom.

Figure 3.

The simulated microcalcification clusters (squared) were overlapped with (a) the uniform background and (b) the tissue structure background, and were acquired with the SF (left), the FP-based (middle) and the CCD-based (right) systems. The MC sizes ranged from 112 to 150 μm for the uniform background and from 200 to 250 μm for the tissue structure background. The matrix sizes for contact (top) and magnified (bottom) images are 1.5 K by 1.5 K, 700 by 700 and 2 K by 2 K for the SF, the FP-based and the CCD-based images, respectively.

Figure 4.

The A_zs for all readers combined are plotted as a function of the average simulated MC size for various imaging system—magnification combinations. The size ranges of 112–150 and 200–250 μm are for the uniform and tissue structure backgrounds, respectively.

Figure 5.

The A_zs for various imaging system—magnification combinations for each individual reader and all readers combined with (a) the uniform background and (b) the tissue structure background. The A_zs were computed for all MC sizes combined.

Figure 6.

The A_zs for different imaging system—magnification combinations for all readers and all MC sizes combined with fibroglandular and heterogeneously dense regions.