Predicting long-term progression of Alzheimer's disease using a multimodal deep learning model incorporating interaction effects

Abstract

Background: Identifying individuals with mild cognitive impairment (MCI) at risk of progressing to Alzheimer's disease (AD) provides a unique opportunity for early interventions. Therefore, accurate and long-term prediction of the conversion from MCI to AD is desired but, to date, remains challenging. Here, we developed an interpretable deep learning model featuring a novel design that incorporates interaction effects and multimodality to improve the prediction accuracy and horizon for MCI-to-AD progression.

Methods: This multi-center, multi-cohort retrospective study collected structural magnetic resonance imaging (sMRI), clinical assessments, and genetic polymorphism data of 252 patients with MCI at baseline from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database. Our deep learning model was cross-validated on the ADNI-1 and ADNI-2/GO cohorts and further generalized in the ongoing ADNI-3 cohort. We evaluated the model performance using the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, and F1 score.

Results: On the cross-validation set, our model achieved superior results for predicting MCI conversion within 4 years (AUC, 0.962; accuracy, 92.92%; sensitivity, 88.89%; specificity, 95.33%) compared to all existing studies. In the independent test, our model exhibited consistent performance with an AUC of 0.939 and an accuracy of 92.86%. Integrating interaction effects and multimodal data into the model significantly increased prediction accuracy by 4.76% (P = 0.01) and 4.29% (P = 0.03), respectively. Furthermore, our model demonstrated robustness to inter-center and inter-scanner variability, while generating interpretable predictions by quantifying the contribution of multimodal biomarkers.

Conclusions: The proposed deep learning model presents a novel perspective by combining interaction effects and multimodality, leading to more accurate and longer-term predictions of AD progression, which promises to improve pre-dementia patient care.

Keywords: Alzheimer’s disease; Artificial intelligence; Deep learning; Early diagnosis; Multimodal biomarkers.

Fig. 1

Flowchart of study design. The study comprised 238 subjects with MCI from ADNI-1 and ADNI-2/GO cohorts for cross-validation, and 14 subjects with MCI from ADNI-3 for an independent test. In addition, 45 subjects diagnosed with AD were included in the model training to address class imbalance

Fig. 2

Schematic illustration of the deep learning model architecture. A The proposed deep learning model consists of multimodal feature extraction and stepwise fusion classification. B Sequential operations within the SepConv block, residual block, and FC block. C Inter-modal and intra-modal interaction modules

Fig. 3

Performance of the deep learning model on the cross-validation set. A Receiver operating characteristic (ROC) curves of tenfold cross-validation. The mean ROC curve with an AUC of 0.962 was obtained by interpolating the ROC curves for tenfolds. Gray shading indicates ± 1 SD of the mean curve. B Confusion matrix of the proposed model on the cross-validation set

Fig. 4

Comparison of model performance with and without interaction effects and multimodality. A Effectiveness evaluation of the dual interaction modules. The performance of our DISFC model was compared with the simple fusion benchmark model without intra-modal and inter-modal interaction modules. The box plot illustrates the 25th percentile (upper box limit), median (horizontal centerline), and 75th percentile (lower box limit). The upper whisker, lower whisker, and hollow circle symbol indicate the maximum, minimum, and mean values of a given model for each metric, respectively. The shaded area on one side around each box represents the probability density. B Performance comparison of the models based on different modality combinations. The unimodal, bimodal, and trimodal models were cross-validated with identical settings. Each bar represents the mean value across folds for each metric

Fig. 5

Comparison of the performance of our proposed model with other state-of-the-art models using the ADNI database. P-values were calculated to compare the performance of previous models with our proposed DISFC model. *P < 0.05; **P < 0.01; ***P < 0.001

Fig. 6

Robustness evaluation across different centers and scanners. A Model performance comparison between clinical sites. B Model performance comparison between scanner manufacturers. C Model performance comparison between scanner magnetic field strengths

Fig. 7

Ablation studies on basic model architecture. A Performance comparison of the models using classical convolution and separable convolution. The models employing classical 3D convolution (Conv3D-based) and separable 3D convolution (SepConv3D-based) were cross-validated with identical settings. Each bar in the chart represents the mean value across folds for respective metrics. B Performance comparison of models based on different fusion schemes. The models based on triple outer product fusion and stepwise fusion were cross-validated under the same settings. Each bar in the chart represents the mean value across folds for respective metrics

Fig. 8

Visualization of the importance of multimodal features. A The top 10 features of most interest to the clinical feature extractor in our model. B The top 10 features of most interest to the genetic feature extractor in our model. C The top 15 brain regions of most interest to the spatial feature extractor in our model, depicted in coronal, axial, and sagittal views for four representative pMCI cases. The color transparency represents the level of importance of the brain region