Machine learning diagnosis of cognitive impairment and dementia in harmonized older adult cohorts

Abstract

Introduction: Clinical diagnosis (normal cognition, mild cognitive impairment [MCI], dementia) is critical for understanding cognitive impairment and dementia but can be resource intensive and subject to inconsistencies due to complex clinical judgments that are required. Machine learning approaches might provide meaningful additions and/or alternatives to traditional clinical diagnosis.

Methods: The study sample was composed of three harmonized longitudinal cohorts of demographically diverse older adults. We used the XGBoost extreme gradient boosting platform to predict clinical diagnosis using different feature sets.

Results: Measures of cognition were especially important predictive features of clinical diagnosis. Prediction accuracy was higher in a sample that had longer follow-up, better balance across diagnostic outcomes, and both self- and informant-report independent function measures.

Discussion: Algorithmic diagnosis might be a meaningful substitute for clinical diagnosis in studies in which clinical evaluation and diagnosis are not feasible for all participants and may provide a standardized alternative when clinical diagnosis is available.

Highlights: A machine learning algorithm was used to diagnose cognitive impairment and dementia. Measures of cognition were strongest predictive features for clinical diagnosis. Algorithm accuracy was improved by informant-report independent function measures. Algorithmic diagnosis might be an alternative if clinical diagnosis is not feasible. Standardization is an important advantage of algorithmic diagnosis.

Keywords: XGBoost algorithm; algorithmic diagnosis; clinical assessment; cognitive impairment; dementia diagnosis; longitudinal cohorts; machine learning; mild cognitive impairment; neuropsychological measures; predictive modeling.

FIGURE 1

Data analysis overview (r = randomly selected). CV, coefficient of variation.

FIGURE 2

Shapley value importance by cognitive impairment outcome (normal, MCI, dementia) for the 20 most influential features. Overall length of bars shows the incremental contribution of each feature to all diagnosis classes, and the colored segments show the contribution to specific diagnosis classes. Results are from the trained Phase 2 model that held out fold 10. aget89, age (top coded at 89); cat, Category Fluency; catch1, Category Fluency Change Lag 1; ecogmem, ECog Informant Memory Summary Score; ecogorg, ECog Informant Organization Summary Score; excheckp, ECog Self “Balance the checkbook without error”; female, female sex/gender; khan, Kaiser Healthy Aging and Diverse Life Experiences Study; la, LatinX race/ethnicity; la90, LifeAfter90 Study; MCI, mild cognitive impairment; memcncrn, ECog Self “Concerned about memory”); mdate, ECog Informant “Remember current date”; mdatep, ECog Self “Remember current date”; phon, Phonemic/Letter Fluency; sem, Semantic Memory; vrmem, Verbal Episodic Memory; vrmemch1, Verbal Episodic Memory Change Lag 1; wh, White race/ethnicity; wm, Working Memory.

FIGURE 3

Shapley dependence plots for verbal episodic memory (vrmem) from assessment concurrent with diagnosis and from 2 previous assessment in a sample with 4+ assessments. (A) shows how SHAP value varies as a function of concurrent vrmem. The color coding of points in (B) and (C) corresponds to the concurrent vrmem values. For example, high concurrent vrmem generally corresponds to less vrmem decline from the previous assessment (negative value of vrmemch1). SHAP, Shapley; vrmem, verbal episodic memory from assessment concurrent with predicted diagnosis; vrmemch1, change in vrmem from 1‐prior assessment to concurrent assessment (1‐prior assessment—concurrent); vrmemch2, change in vrmem from 2‐prior assessment to concurrent assessment (2‐prior assessment—concurrent).

FIGURE 4

Shapley waterfall plots for two groups with clinical diagnoses of dementia, one composed of individuals ≤ 70 years of age and the second of individual in the 85+ range. These plots show averaged independent contributions of features to the outcome (log‐odds of dementia) in each group. f(x) ∼ average log‐odds of dementia in depicted groups (corresponding probabilities of dementia: ≤ 70 = 0.90, ≥ 85 = 0.82), E(f[x]) ∼ average log‐odds of dementia in full sample (corresponding probability of dementia = 0.38). aget89, age (top coded at 89); cat, Category Fluency; ecogmem, ECog Informant Memory Summary Score; ecogorg, ECog Informant Organization Summary Score; excheckp, ECog Self “Balance the checkbook without error”; memcncrn, ECog Self “Concerned about memory”); mdate, ECog Informant “Remember current date”; mdatep, ECog Self “Remember current date”; phon, Phonemic/Letter Fluency; sem, Semantic Memory; vrmem, Verbal Episodic Memory; wm, Working Memory.

FIGURE 5

ROC curves for study cohorts comparing clinical diagnosis with the out‐of‐sample prediction of clinical diagnosis. Plot A shows results for a diagnosis of dementia versus non‐dementia (normal cognition or MCI), Plot B is for a diagnosis of cognitive impairment (MCI or dementia) versus normal cognition. AUC, ROC area under the curve (95% confidence interval). ADRC, Alzheimer's Disease Research Center; AUC, area under the curve; KHANDLE, Kaiser Healthy Aging and Diverse Life Experiences Study; LA90, LifeAfter90 Study; MCI, mild cognitive impairment; ROC, receiver operating characteristic.