Early prediction of neoadjuvant chemotherapy response by exploiting a transfer learning approach on breast DCE-MRIs

Abstract

The dynamic contrast-enhanced MR imaging plays a crucial role in evaluating the effectiveness of neoadjuvant chemotherapy (NAC) even since its early stage through the prediction of the final pathological complete response (pCR). In this study, we proposed a transfer learning approach to predict if a patient achieved pCR (pCR) or did not (non-pCR) by exploiting, separately or in combination, pre-treatment and early-treatment exams from I-SPY1 TRIAL public database. First, low-level features, i.e., related to local structure of the image, were automatically extracted by a pre-trained convolutional neural network (CNN) overcoming manual feature extraction. Next, an optimal set of most stable features was detected and then used to design an SVM classifier. A first subset of patients, called fine-tuning dataset (30 pCR; 78 non-pCR), was used to perform the optimal choice of features. A second subset not involved in the feature selection process was employed as an independent test (7 pCR; 19 non-pCR) to validate the model. By combining the optimal features extracted from both pre-treatment and early-treatment exams with some clinical features, i.e., ER, PgR, HER2 and molecular subtype, an accuracy of 91.4% and 92.3%, and an AUC value of 0.93 and 0.90, were returned on the fine-tuning dataset and the independent test, respectively. Overall, the low-level CNN features have an important role in the early evaluation of the NAC efficacy by predicting pCR. The proposed model represents a first effort towards the development of a clinical support tool for an early prediction of pCR to NAC.

Figure 1

Chart of the MRI acquisitions for patients of the I-SPY1 TRIAL dataset undergoing neoadjuvant chemotherapy. Pre-treatment and early-treatment exams (MRI at T1 and MRI at T2, respectively) were analysed for the prediction of pathological Complete Response (pCR). The final treatment evaluation was performed at the end of chemotherapy and after the surgery by pCR.

Figure 2

ROC curves for pCR prediction models. (a) ROC curves related to the model with the Optimal Subset of Features (OSF) at timepoints T1 and T2 separately or in combination and evaluated on the fine-tuning dataset. (b) ROC curves related to the best model (OSF at T1-T2) and the best model with clinical variables (OSF at T1-T2 + clinical) and evaluated on the independent test. (a, b) The AUC values are also highlighted.

Figure 3

(a) Activation maps of the ROIs of both MRI at T1 and MRI at T2 from which the selected optimal features were extracted. The red squares outline with more precision the area of belonging of such features. Each ROI has dimensions of 227 $\times$ 227 pixels. Each activation map has original dimensions of 13 $\times$ 13 pixels and has been resized to the dimensions of the corresponding ROIs for a better visualization. (b) Convolutional maps related to the activation maps of the ROIs of both MRI at T1 and MRI at T2 from which the selected optimal features were extracted. Each ROI has dimensions of 227 $\times$ 227 pixels. Each convolutional map has original dimensions of 27 $\times$ 27 pixels and has been resized to the dimensions of the corresponding ROI for a better visualization.

Figure 4

Workflow of the proposed transfer learning method encompassing five steps: 1. Feature extraction, 2. Dynamic feature selection, 3. Optimal feature selection, 4. Classification on the fine-tuning dataset, 5. Classification on the independent test.