Predicting the Efficacy of Neoadjuvant Chemotherapy for Pancreatic Cancer Using Deep Learning of Contrast-Enhanced Ultrasound Videos

Abstract

Contrast-enhanced ultrasound (CEUS) is a promising imaging modality in predicting the efficacy of neoadjuvant chemotherapy for pancreatic cancer, a tumor with high mortality. In this study, we proposed a deep-learning-based strategy for analyzing CEUS videos to predict the prognosis of pancreatic cancer neoadjuvant chemotherapy. Pre-trained convolutional neural network (CNN) models were used for binary classification of the chemotherapy as effective or ineffective, with CEUS videos collected before chemotherapy as the model input, and with the efficacy after chemotherapy as the reference standard. We proposed two deep learning models. The first CNN model used videos of ultrasound (US) and CEUS (US+CEUS), while the second CNN model only used videos of selected regions of interest (ROIs) within CEUS (CEUS-ROI). A total of 38 patients with strict restriction of clinical factors were enrolled, with 76 original CEUS videos collected. After data augmentation, 760 and 720 videos were included for the two CNN models, respectively. Seventy-six-fold and 72-fold cross-validations were performed to validate the classification performance of the two CNN models. The areas under the curve were 0.892 and 0.908 for the two models. The accuracy, recall, precision and F1 score were 0.829, 0.759, 0.786, and 0.772 for the first model. Those were 0.864, 0.930, 0.866, and 0.897 for the second model. A total of 38.2% and 40.3% of the original videos could be clearly distinguished by the deep learning models when the naked eye made an inaccurate classification. This study is the first to demonstrate the feasibility and potential of deep learning models based on pre-chemotherapy CEUS videos in predicting the efficacy of neoadjuvant chemotherapy for pancreas cancer.

Keywords: contrast-enhanced ultrasound; deep learning; neoadjuvant chemotherapy; pancreatic cancer; prognosis prediction.

Figure 1

Flow chart of patient enrollment and data preprocessing. BRPC: borderline resectable pancreatic cancer; LAPC: locally advanced pancreatic cancer; NACT: neoadjuvant chemotherapy; S-1: tegafur, gimeracil and oteracil potassium; US: ultrasound; CEUS: contrast-enhanced ultrasound; ROI: region of interest.

Figure 2

US+CEUS and CEUS-ROI images. (A,B) An example of typical US+CEUS frames of 20 s (arterial phase) (A) and 60 s (venous phase) (B) of a patient with ineffective treatment outcome, showing hypo-enhancement pattern. (C,D) An example of typical CEUS-ROI frame of 20 s (arterial phase) (C) and 60 s (venous phase) (D) of a patient with effective treatment outcome, showing iso-enhancement pattern. The white arrow indicates pancreas cancer. US: ultrasound; CEUS: contrast-enhanced ultrasound; ROI: region of interest.

Figure 3

(A) Flow chart of the proposed deep learning method. (B) Data augmentation. (C) n-fold cross-validation.

Figure 4

ROC curves obtained by using the two deep learning strategies with US+CEUS (A) and CEUS-ROI (B).

Figure 5

Prediction of each original video by US+CEUS. (A) 2 × 2 contingency table of results of final predictions of DL and in clinical practice. (B) Venn Diagram of the number of cases that was classified as C-EF, NE-EF, and DL-EF. (C) Venn Diagram of the number of cases that was classified as C-INEF, NE-INEF, and DL-INEF. C-(IN)EF: clinically (in)effective; DL-(IN)EF: deep learning predicted (in)effectiveness; NE-(IN)EF: naked eye predicted (in)effectiveness.

Figure 6

Prediction of each original video by CEUS-ROI. (A) 2 × 2 contingency table of results of final predictions of DL and in clinical practice. (B) Venn diagram of the number of cases that was classified as C-EF, NE-EF, and DL-EF. (C) Venn diagram of the number of cases that was classified as C-INEF, NE-INEF, and DL-INEF. C-(IN)EF: clinically (in)effective; DL-(IN)EF: deep learning predicted (in)effectiveness; NE-(IN)EF: naked eye predicted (in)effectiveness.