Mammography-based artificial intelligence for breast cancer detection, diagnosis, and BI-RADS categorization using multi-view and multi-level convolutional neural networks

Abstract

Purpose: We developed an artificial intelligence system (AIS) using multi-view multi-level convolutional neural networks for breast cancer detection, diagnosis, and BI-RADS categorization support in mammography.

Methods: Twenty-four thousand eight hundred sixty-six breasts from 12,433 Asian women between August 2012 and December 2018 were enrolled. The study consisted of three parts: (1) evaluation of AIS performance in malignancy diagnosis; (2) stratified analysis of BI-RADS 3-4 subgroups with AIS; and (3) reassessment of BI-RADS 0 breasts with AIS assistance. We further evaluate AIS by conducting a counterbalance-designed AI-assisted study, where ten radiologists read 1302 cases with/without AIS assistance. The area under the receiver operating characteristic curve (AUC), sensitivity, specificity, accuracy, and F1 score were measured.

Results: The AIS yielded AUC values of 0.995, 0.933, and 0.947 for malignancy diagnosis in the validation set, testing set 1, and testing set 2, respectively. Within BI-RADS 3-4 subgroups with pathological results, AIS downgraded 83.1% of false-positives into benign groups, and upgraded 54.1% of false-negatives into malignant groups. AIS also successfully assisted radiologists in identifying 7 out of 43 malignancies initially diagnosed with BI-RADS 0, with a specificity of 96.7%. In the counterbalance-designed AI-assisted study, the average AUC across ten readers significantly improved with AIS assistance (p = 0.001).

Conclusion: AIS can accurately detect and diagnose breast cancer on mammography and further serve as a supportive tool for BI-RADS categorization.

Critical relevance statement: An AI risk assessment tool employing deep learning algorithms was developed and validated for enhancing breast cancer diagnosis from mammograms, to improve risk stratification accuracy, particularly in patients with dense breasts, and serve as a decision support aid for radiologists.

Key points: The false positive and negative rates of mammography diagnosis remain high. The AIS can yield a high AUC for malignancy diagnosis. The AIS is important in stratifying BI-RADS categorization.

Keywords: Artificial intelligence; Breast neoplasms; Deep learning; Diagnosis; Mammography.

Fig. 1

The workflow of study inclusion and exclusion

Fig. 2

The workflow of the study. A The first step of the AIS. B The second step of the AIS. C The workflow of the AI assistance study. BI-RADS, breast imaging reporting and data system; AIS, artificial intelligence system

Fig. 3

Stratified analysis of gland type and lesion type, and cancer type. A A stratified analysis of gland type. B A stratified analysis of lesion type. C A stratified analysis of cancer type

Fig. 4

Stratified analysis of BI-RADS 3–4 subgroups. A The classification performance of the AIS and BI-RADS lexicon within the BI-RADS 3–4 subgroups. B The sunburst plot of the combined results in the validation set, testing set 1, and testing set 2. The inner ring represents the breast distribution of the BI-RADS 3, 4A, 4B, and 4C. Data in parentheses are a number of cases. The second ring represents the pathology distribution; dark blue means benign cases, and dark red means malignant cases. The third ring represents the prediction distribution from the AIS; blue means benign prediction, and red means malignant prediction. For example, 1439 of the breasts were BI-RADS 4B, in which 449 were benign cases and 990 were malignant, proven by pathology. The AIS can correctly classify 369 out of 449 benign cases into the benign group. For these 369 cases, the AIS can benefit them and avoid over-treatment through biopsy. In the meantime, 146 cases out of 990 malignant cases were misclassified into the benign group. Overall, the benefit cases are more than the missed cases. BI-RADS, breast imaging reporting and data system; ROC, receiver operating characteristic; AIS, artificial intelligence system

Fig. 5

The results of the AI assistance study. Readers 1 and 2 are junior group (standardization training residents without any experience in mammography diagnosis), readers 3–6 are mid-level group (radiologists with 3–10 years of mammography diagnostic experience), and readers 7–10 are senior group (radiologists with more than 10 years of mammography diagnostic experience)

Fig. 6

Two representative dense breasts in the AI assistance study. Patient 1 has benign breast cancer, and Patient 2 has malignant breast cancer. Patients 1 and 2 are both dense parenchyma breasts that radiologists often struggle to classify as benign or malignant. But with the assistance of AIS, most readers can tell them apart. Patient 1: mammograms for a 48-year-old woman with adenosis showed a dense gland with structural distortion in the lower outer quadrant of the left breast. With the help of AIS, the BI-RADS categories for ten readers were adjusted accordingly from 4C, 4B, 3, 4B, 4C, 3, 1, 3, 4A, 4B to 1, 4B, 3, 4B, 3, 3, 1, 3, 3, 3, respectively. Patient 2: mammograms for a 42-year-old woman with IDC showed dense gland, patchy high-density shadow, and coarse spotty calcifications in the lower inner quadrant of the left breast. With the help of AI, the BI-RADS categories for ten readers were adjusted accordingly from 3, 4B, 3, 3, 4B, 4B, 2, 4A, 3, 3 to 4C, 4C, 3, 4B, 2, 4A, 4A, 4B, 4B, 3, respectively. BI-RADS, breast imaging reporting and data system; AIS, artificial intelligence system