A highly predictive signature of cognition and brain atrophy for progression to Alzheimer's dementia

Abstract

Background: Clinical trials in Alzheimer's disease need to enroll patients whose cognition will decline over time, if left untreated, in order to demonstrate the efficacy of an intervention. Machine learning models used to screen for patients at risk of progression to dementia should therefore favor specificity (detecting only progressors) over sensitivity (detecting all progressors), especially when the prevalence of progressors is low. Here, we explore whether such high-risk patients can be identified using cognitive assessments and structural neuroimaging by training machine learning tools in a high-specificity regime.

Results: A multimodal signature of Alzheimer's dementia was first extracted from the ADNI1 dataset. We then validated the predictive value of this signature on ADNI1 patients with mild cognitive impairment (N = 235). The signature was optimized to predict progression to dementia over 3 years with low sensitivity (55.1%) but high specificity (95.6%), resulting in only moderate accuracy (69.3%) but high positive predictive value (80.4%, adjusted for a "typical" 33% prevalence rate of true progressors). These results were replicated in ADNI2 (N = 235), with 87.8% adjusted positive predictive value (96.7% specificity, 47.3% sensitivity, 85.1% accuracy).

Conclusions: We found that cognitive measures alone could identify high-risk individuals, with structural measurements providing a slight improvement. The signature had comparable receiver operating characteristics to standard machine learning tools, yet a marked improvement in positive predictive value was achieved over the literature by selecting a high-specificity operating point. The multimodal signature can be readily applied for the enrichment of clinical trials.

Keywords: Alzheimer's disease; cognition; machine learning; mild cognitive impairment; neuroimaging.

Figure 1:

Subtyping procedure and resulting subtypes. (a) A hierarchical clustering procedure identified 7 subtypes, or subgroups, of individuals with similar patterns of grey matter topography within the ADNI1 cohort of CN and AD participants (top). A measure of spatial similarity, called subtype weight, between a single individual's GMV map and the average of a given subtype was calculated for all individuals and all subtypes (bottom). (b) Maps of the 7 subtypes showing the distribution of grey matter across all voxels relative to the average. CN* and AD* denote significant associations between the subtype weights and diagnoses of CN or AD, respectively.

Figure 2:

Accuracy, specificity, sensitivity, and PPV for different classifiers: linear SVM, HPS, and the linear SVM thresholded at 0.95 (SVM 0.95), for the classifications of patients with AD and CN individuals and patients with MCI who progress to AD (pMCI) and stable MCI (sMCI) in ADNI1 and ADNI2. VBM represents the model trained with VBM subtypes, COG represents the model trained with baseline cognitive scores, and VCOG represents the model trained with both VBM subtypes and cognition. PPV was adjusted [PPV (adj)] for a prevalence of 33.6% pMCI in a sample of MCI participants for both ADNI1 and ADNI2 MCI cohorts. Significant differences are denoted by * for P < 0.05 and ** for P < 0.001.

Figure 3:

ROC curves for various machine learning algorithms with different features (VBM for VBM subtypes only, COG for cognitive features only, VCOG for a combination of VBM subtypes and cognitive features). Algorithms included an SVM with a radial basis function kernel (RBF SVM), K nearest neighbours (KNN), random forest (RF), Gaussian naive Bayes (GNB), an SVM with a linear kernel representing the first stage (Linear SVM) of the 2-stage predictive model, and the 2-stage HPS. TPR: true-positive rate; FPR: false-positive rate; AUC: area under the curve.

Figure 4:

Characteristics of participants with MCI with the VCOG signature in ADNI1 and ADNI2. We show the percentage of participants with MCI who (a) progressed to dementia and were (b) APOE4 carriers, (c) female, (d) positive for Aβ measured by a cut-off of 192 pg/mL in the CSF [38], and (e) positive for τ measured by a cut-off of 93 pg/mL in the CSF [38] in each classification (high confidence, low confidence, and negative). (f) Age and (g) cognitive trajectories, measured by the Alzheimer's Disease Assessment Scale-Cognitive subscale with 13 items (ADAS13), across the 3 classes. Significant differences are denoted by asterisks for family-wise error rate-corrected P < 0.05.

Figure 5:

Coefficients of the high-confidence prediction (a) VCOG HPS model, (b) COG HPS model, (c) VBM HPS model. ADAS13 = Alzheimer's Disease Assessment Scale—Cognitive; MEM = ADNI-MEM score; EXEC = ADNI-EF score; BNT = Boston Naming Test; CLOCK = clock drawing test; VBM 1–7 = VBM subtype weights.

Figure 6:

Venn diagram depicting the number of participants with MCI labelled as high confidence by the VBM, COG, and VCOG HPS models in ADNI1 and ADNI2.