Deep learning-enabled fully automated pipeline system for segmentation and classification of single-mass breast lesions using contrast-enhanced mammography: a prospective, multicentre study

Abstract

Background: Breast cancer is the leading cause of cancer-related deaths in women. However, accurate diagnosis of breast cancer using medical images heavily relies on the experience of radiologists. This study aimed to develop an artificial intelligence model that diagnosed single-mass breast lesions on contrast-enhanced mammography (CEM) for assisting the diagnostic workflow.

Methods: A total of 1912 women with single-mass breast lesions on CEM images before biopsy or surgery were included from June 2017 to October 2022 at three centres in China. Samples were divided into training and validation sets, internal testing set, pooled external testing set, and prospective testing set. A fully automated pipeline system (FAPS) using RefineNet and the Xception + Pyramid pooling module (PPM) was developed to perform the segmentation and classification of breast lesions. The performances of six radiologists and adjustments in Breast Imaging Reporting and Data System (BI-RADS) category 4 under the FAPS-assisted strategy were explored in pooled external and prospective testing sets. The segmentation performance was assessed using the Dice similarity coefficient (DSC), and the classification was assessed using heatmaps, area under the receiver operating characteristic curve (AUC), sensitivity, and specificity. The radiologists' reading time was recorded for comparison with the FAPS. This trial is registered with China Clinical Trial Registration Centre (ChiCTR2200063444).

Findings: The FAPS-based segmentation task achieved DSCs of 0.888 ± 0.101, 0.820 ± 0.148 and 0.837 ± 0.132 in the internal, pooled external and prospective testing sets, respectively. For the classification task, the FAPS achieved AUCs of 0.947 (95% confidence interval [CI]: 0.916-0.978), 0.940 (95% [CI]: 0.894-0.987) and 0.891 (95% [CI]: 0.816-0.945). It outperformed radiologists in terms of classification efficiency based on single lesions (6 s vs 3 min). Moreover, the FAPS-assisted strategy improved the performance of radiologists. BI-RADS category 4 in 12.4% and 13.3% of patients was adjusted in two testing sets with the assistance of FAPS, which may play an important guiding role in the selection of clinical management strategies.

Interpretation: The FAPS based on CEM demonstrated the potential for the segmentation and classification of breast lesions, and had good generalisation ability and clinical applicability.

Funding: This study was supported by the Taishan Scholar Foundation of Shandong Province of China (tsqn202211378), National Natural Science Foundation of China (82001775), Natural Science Foundation of Shandong Province of China (ZR2021MH120), and Special Fund for Breast Disease Research of Shandong Medical Association (YXH2021ZX055).

Keywords: Breast lesions; Classification; Contrast-enhanced mammography; Deep learning; Full automated pipeline system; Segmentation.

Fig. 1

Study profile.

Fig. 2

The design of this study. (A) Process of FAPS to perform segmentation and classification tasks. (B) The process in which FAPS assisted radiologists in pooled external and prospective testing sets. The blue arrows represent patients whose BI-RADS categories were upgraded, and the red ones were downgraded. FAPS = fully automated pipeline system; PPM = pyramid pooling module; CC = craniocaudal; MLO = medio lateral-oblique; BI-RADS = Breast Imaging Reporting and Data System.

Fig. 3

Performance of the segmentation network. (A) The performance of the Xception segmentation model based on the DSC on the training and validation sets, as well as the internal testing set. (B) Visualisation of segmentation results. (a) and (e) are the low-energy images of the CC view and MLO view, respectively. (b) and (f) are the recombined images of the CC view and MLO view, respectively. (c) and (g) are the real masks, and (d) and (h) are the segmentation results. The above images in a 36-year-old woman with fibroadenoma. CEM images show a 6.4 cm lesion. The DSC is 0.955 for the CC view and 0.954 for the MLO view. Images below in a 56-year-old woman with phyllodes tumor. CEM images show a 3.5 cm lesion. DSC is 0.906 for the CC view and 0.936 the for MLO view. CC = craniocaudal; MLO = medio lateral-oblique; CEM = contrast-enhanced mammography; DSC = dice similarity coefficient.

Fig. 4

The performance of the FAPS and radiologists without and with FAPS assistance in the pooled external testing set (A) and prospective testing set (B). ROC = receiver operating characteristic; AUC = area under receiver operating characteristic curve; FAPS = fully automated pipeline system; R = radiologist.

Fig. 5

The agreement degree of pairs of radiologists without and with FAPS assistance in the pooled external testing set (A and B) and prospective testing set (C and D). FAPS = fully automated pipeline system.

Fig. 6

Subgroup analysis in the pooled external and prospective testing sets. Sensitivity and specificity with or without FAPS-assisted diagnosis in the dense breast subgroup (A and B) and the lesion diameter ≤2 cm subgroup (C and D). FAPS = fully automated pipeline system.

Fig. 7

Heatmap analysis of four controversial cases. The three images above are the low-energy image, the recombined image (b), and the heatmap of the CC view (c). The red regions have higher predictive significance than the green and blue regions. (A) Images in a 36-year-old woman with fibroadenoma. CEM images show a 6.4 cm lesion. (B) Images in a 56-year-old woman with a phyllodes tumor. CEM images show a 3.5 cm lesion. (C) Images in a 53-year-old woman with invasive ductal carcinoma. CEM images show a 2.2 cm lesion. (D) Images in a 63-year-old woman with invasive ductal carcinoma. CEM images show a 4.8 cm lesion. FAPS = fully automated pipeline system; BI-RADS = Breast Imaging Reporting and Data System; CC = craniocaudal; MLO = medio lateral-oblique; R = radiologist; CEM = contrast-enhanced mammography.