Deep Learning to Distinguish Recalled but Benign Mammography Images in Breast Cancer Screening

Abstract

Purpose: False positives in digital mammography screening lead to high recall rates, resulting in unnecessary medical procedures to patients and health care costs. This study aimed to investigate the revolutionary deep learning methods to distinguish recalled but benign mammography images from negative exams and those with malignancy.

Experimental design: Deep learning convolutional neural network (CNN) models were constructed to classify mammography images into malignant (breast cancer), negative (breast cancer free), and recalled-benign categories. A total of 14,860 images of 3,715 patients from two independent mammography datasets: Full-Field Digital Mammography Dataset (FFDM) and a digitized film dataset, Digital Dataset of Screening Mammography (DDSM), were used in various settings for training and testing the CNN models. The ROC curve was generated and the AUC was calculated as a metric of the classification accuracy.

Results: Training and testing using only the FFDM dataset resulted in AUC ranging from 0.70 to 0.81. When the DDSM dataset was used, AUC ranged from 0.77 to 0.96. When datasets were combined for training and testing, AUC ranged from 0.76 to 0.91. When pretrained on a large nonmedical dataset and DDSM, the models showed consistent improvements in AUC ranging from 0.02 to 0.05 (all P > 0.05), compared with pretraining only on the nonmedical dataset.

Conclusions: This study demonstrates that automatic deep learning CNN methods can identify nuanced mammographic imaging features to distinguish recalled-benign images from malignant and negative cases, which may lead to a computerized clinical toolkit to help reduce false recalls.

Figure 1.

Performance results for deep learning convolutional neural network (CNN) models for classification on the full-field digital mammography (FFDM) dataset. (Left) Receiver operating characteristic (ROC) curves for the binary classification scenarios and corresponding area under the curves (AUC). (Right) ROC curves for the triple-class classification scenario and averaged AUC.

Figure 2.

Performance results for deep learning convolutional neural network (CNN) models for classification on the digital database of screening mammography (DDSM) dataset. (Left) Receiver operating characteristic (ROC) curves for the binary classification scenarios and corresponding area under the curves (AUC). (Right) ROC curves for the triple-class classification scenario and averaged AUC.

Figure 3.

Performance results for deep learning convolutional neural network (CNN) models for classification using combined Full-field Digital Mammography (FFDM) and digital database of screening mammography (DDSM) datasets for training and testing. (Left) Receiver operating characteristic (ROC) curves for the binary classification scenarios and corresponding area under the curves (AUC). (Right) ROC curves for the triple-class classification scenario and averaged AUC.

Figure 4.

Comparison of performance results of deep learning convolutional neural network (CNN) models on different pre-training strategies: using original ImageNet pre-trained model vs. using model pre-trained on ImageNet and digital database of screening mammography (DDSM) dataset. All the area under the curves (AUCs) were results based on training (fine-tuning) and testing on the full-field digital mammography (FFDM) dataset.

Figure 5.

Comparison of performance results using varying amounts (5%, 10%, and 15%) of testing data across all models: (Top-Left) FFDM trained model; (Top-Right) DDSM trained model; (Bottom-Left) FFDM + DDSM trained model; (Bottom-Right) Incrementally pre-trained CNN models in all scenarios.