Prediction of axillary lymph node metastasis in early breast cancer patients with ultrasonic videos based deep learning

Abstract

Objective: To develop a deep learning (DL) model for predicting axillary lymph node (ALN) metastasis using dynamic ultrasound (US) videos in breast cancer patients.

Methods: A total of 271 US videos from 271 early breast cancer patients collected from Xiang'an Hospital of Xiamen University andShantou Central Hospitabetween September 2019 and June 2021 were used as the training, validation, and internal testing set (testing set A). Additionally, an independent dataset of 49 US videos from 49 patients with breast cancer, collected from Shanghai 10th Hospital of Tongji University from July 2021 to May 2022, was used as an external testing set (testing set B). All ALN metastases were confirmed using pathological examination. Three different convolutional neural networks (CNNs) with R2 + 1D, TIN, and ResNet-3D architectures were used to build the models. The performance of the US video DL models was compared with that of US static image DL models and axillary US examination performed by ultra-sonographers. The performances of the DL models and ultra-sonographers were evaluated based on accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUC). Additionally, gradient class activation mapping (Grad-CAM) technology was also used to enhance the interpretability of the models.

Results: Among the three US video DL models, TIN showed the best performance, achieving an AUC of 0.914 (95% CI: 0.843-0.985) in predicting ALN metastasis in testing set A. The model achieved an accuracy of 85.25% (52/61), with a sensitivity of 76.19% (16/21) and a specificity of 90.00% (36/40). The AUC of the US video DL model was superior to that of the US static image DL model (0.856, 95% CI: 0.753-0.959, P<0.05). The Grad-CAM technology confirmed the heatmap of the model, which highlighted important subregions of the keyframe for ultra-sonographers' review.

Conclusion: A feasible and improved DL model to predict ALN metastasis from breast cancer US video images was developed. The DL model in this study with reliable interpretability would provide an early diagnostic strategy for the appropriate management of axillary in the early breast cancer patients.

Keywords: artificial intelligence; axillary lymph node metastasis; breast lesion; deep learning model; ultrasound video image.

Figure 1

Patient recruitment workflow. In total, 271 US videos of 271 patients were included according to the inclusion criteria. The included patients were examined by conventional US and had complete clinical information for this study.

Figure 2

The overview of our DL model architecture. It mainly consists of three modules including feature extraction, feature fusion, and final classification prediction.

Figure 3

The visual class activation diagram of the prediction results of dynamic video models. The brighter the color, the better the visibility of the model. The best TIN model focuses more on the location of the nodules than the other two models, and the characteristics of the nodules are the most important basis for predicting whether the nodules will metastasize.

Figure 4

The figure shows the change curve of the loss function during the training of the dynamic video model. Subfigure (A, B) are Acc and loss curve for 300 training epoches. Subfigure (C, D) are Acc and loss curve for 300 validating epoches. TIN and ResNet converged well and stabilized at a small value of 0.2-0.3 in 300 training rounds. R2 + 1D converges relatively, but the trend is not obvious enough and the convergence value is about 0.6, which is a big gap. This also explains why its results on the test set are significantly lower than those of the other two models.

Figure 5

Comparison of receiver operating characteristic (ROC) curves between two models (video vs. static image) for predicting ALN metastasis.

Figure 6

Comparison of receiver operating characteristic (ROC) curves between the ultra-sonographers and DL model.

Figure 7

The visual activation diagram of the prediction results of the static image model. The brighter the place, the higher the model’s attention. It can be seen that all models basically pay attention to the characteristics of the nodule area, but they are not concentrated enough. Among them, Inception V3 and VGG19 models focus on the left side of the nodule, which also affects the final prediction results to some extent.