Transfer learning for predicting conversion from mild cognitive impairment to dementia of Alzheimer's type based on a three-dimensional convolutional neural network

Abstract

Dementia of Alzheimer's type (DAT) is associated with devastating and irreversible cognitive decline. Predicting which patients with mild cognitive impairment (MCI) will progress to DAT is an ongoing challenge in the field. We developed a deep learning model to predict conversion from MCI to DAT. Structural magnetic resonance imaging scans were used as input to a 3-dimensional convolutional neural network. The 3-dimensional convolutional neural network was trained using transfer learning; in the source task, normal control and DAT scans were used to pretrain the model. This pretrained model was then retrained on the target task of classifying which MCI patients converted to DAT. Our model resulted in 82.4% classification accuracy at the target task, outperforming current models in the field. Next, we visualized brain regions that significantly contribute to the prediction of MCI conversion using an occlusion map approach. Contributory regions included the pons, amygdala, and hippocampus. Finally, we showed that the model's prediction value is significantly correlated with rates of change in clinical assessment scores, indicating that the model is able to predict an individual patient's future cognitive decline. This information, in conjunction with the identified anatomical features, will aid in building a personalized therapeutic strategy for individuals with MCI.

Keywords: Convolutional neural network; Dementia of Alzheimer's type; Magnetic resonance imaging; Mild cognitive impairment; Predictive modeling.

Figure 1.

Distribution of MCI-C (N=277) and MCI-NC (N=514) patients according to years until conversion and duration of diagnosis, respectively: Upper histogram red box: MCI-C patients who converted to DAT within 3 years were selected for the target task (N=228). As a comparison to this group, lower histogram red box: MCI-NC patients whose duration of MCI diagnosis is at least 3 years (N=222) are included in this study.

Figure 2.

Graphical layout of data division for the source task and target task. The number of scans and patients used for train, validation, and test set are shown. The percentage in parenthesis indicates the ratio of data size compared to the whole dataset, i.e., either source or target data set. The number of 1.5T and 3.0T scans are also shown in brackets.

Figure 3.

(a). Architecture of Convolutional Neural Network (CNN). The original ImageNet Model, i.e., ResNet50 was scaled down by narrowing and shortening the model. Figure 3. (b) Bottleneck layers were set to reduce the model’s complexity and thereby improve the classification performance (He et al., 2016). Figure 3. (c) Skip connection was used to enable the model to reach a global optima (He et al., 2016).

Figure 4.

Loss history of train and validation data (a) and classification performance (b), i.e., Area Under the Curve (AUC) and Equal Error Rate (EER) on test data. Train and validation loss continuously decreased along the epochs, indicating that the model was learning. The weight matrix that was restored and used to evaluate the test classification accuracy was where the validation loss showed the minimum. Test classification accuracy reported 82.4%. AUC and EER value are 0.827 and 0.189, respectively.

Figure 5.

The sensitivity to predict patients with conversion years from 0 to 10. The random guess is 10% as it is the chance of one out of 10 different conversion years.

Figure 6.

Occlusion maps (a) sagittal plane, (b) coronal plane, and (c) transverse plane across all correctly predicted MCI-NC patients. The red color indicates a higher prediction score, whereas the blue color indicates a lower prediction score. The blue regions indicate importance in predicting MCI-NC, and include the pons, amygdala, hippocampus, and parahippocampal gyrus.

Figure 7.

Occlusion maps (a) sagittal plane, (b) coronal plane, and (c) transverse plane across all correctly predicted MCI-C patients. The red color indicates a higher prediction score, whereas the blue color indicates a lower prediction score. The blue regions indicate importance in predicting MCI-C, and include the midbrain, nucleus accumbens, caudate nucleus, cerebellum, globus pallidus, and thalamus.