Applying Automated MR-Based Diagnostic Methods to the Memory Clinic: A Prospective Study

Abstract

Several studies have demonstrated that fully automated pattern recognition methods applied to structural magnetic resonance imaging (MRI) aid in the diagnosis of dementia, but these conclusions are based on highly preselected samples that significantly differ from that seen in a dementia clinic. At a single dementia clinic, we evaluated the ability of a linear support vector machine trained with completely unrelated data to differentiate between Alzheimer's disease (AD), frontotemporal dementia (FTD), Lewy body dementia, and healthy aging based on 3D-T1 weighted MRI data sets. Furthermore, we predicted progression to AD in subjects with mild cognitive impairment (MCI) at baseline and automatically quantified white matter hyperintensities from FLAIR-images. Separating additionally recruited healthy elderly from those with dementia was accurate with an area under the curve (AUC) of 0.97 (according to Fig. 4). Multi-class separation of patients with either AD or FTD from other included groups was good on the training set (AUC > 0.9) but substantially less accurate (AUC = 0.76 for AD, AUC = 0.78 for FTD) on 134 cases from the local clinic. Longitudinal data from 28 cases with MCI at baseline and appropriate follow-up data were available. The computer tool discriminated progressive from stable MCI with AUC = 0.73, compared to AUC = 0.80 for the training set. A relatively low accuracy by clinicians (AUC = 0.81) illustrates the difficulties of predicting conversion in this heterogeneous cohort. This first application of a MRI-based pattern recognition method to a routine sample demonstrates feasibility, but also illustrates that automated multi-class differential diagnoses have to be the focus of future methodological developments and application studies.

Keywords: Dementia diagnostics; machine learning; magnetic resonance imaging; prognosis; support vector machine.

Fig.1

Distribution of patients entering the memory clinic and their inclusion in the differential diagnosing of dementia (left panel) and predicting of MCI conversion (right panel). LBD, Lewy body dementia; FTD, frontotemporal dementia; AD, Alzheimer’s disease; HC, healthy controls; VaD, vascular dementia; MCI, mild cognitive impairment.

Fig.2

Histograms showing increased levels of diagnostic confidence for the prediction of conversion from MCI (x-axis) by clinicians after learning about the MRI-results. In addition, a separation into cases correctly (green) or incorrectly (red) predicted by the SVM indicates no association between the diagnostic confidence of clinicians and the accuracy of the SVM.

Fig.3

Separating stable (MCIs) from those converting to dementia (MCIc). Left: ROC curve for different levels of diagnostic confidence (clinicians) and decision values (SVM). The cross-validated SVM performance on train set (dashed black line), test set (solid blue line) and performance by clinicians at post-MRI (dotted light blue) is shown. FPR, false positive rate; TPR, true positive rate. Right: True positive (TPR) and negative rate (TNR) together with positive (PPV) and negative predictive value (NPV). Markers on the x-axis indicate individual cases: green circles: MCIs; red crosses: MCIc. p(MCIs) and p(MCIc) indicate the fraction of stable and progressive MCI subjects in the sample, respectively.

Fig.4

Performance of multiclass differential diagnosis of dementia. The top row displays the ROC curve of each class versus the rest. Dotted black line and solid blue line indicate cross-validated training using cross-validation and test performance respectively. The bottom row shows several performance measure such as positive predictive value (PPV), negative predictive value (NPV), true positive rate (TPR), and true negative rate (TNR). See main text for AUCs of the training set.

Fig.5

Performance of differential diagnosis of FTD versus AD. Left: ROC curve, where the dashed black line indicates the cross-validated result from the training data and the solid blue line the test result. Red and green dashed lines illustrate the performance when cases are restricted to those with high quality (HQ) or cases without comorbid brain disorders (no comorb). Right: Indication of usefulness in terms of PPV and NPV. Markers on the x-axis indicate FTD (green circles) and AD (red crosses).

Fig.6

Radar plot illustrating the posterior probability for each diagnostic class and the proportion of white matter hyperintensities (WMH). Center hexagon indicates minimum diagnostic probability or WMH load. Each line represents one case. All cases with dementia are shown in each subplot but with intransparent lines for a different class to aid visualization. Lines representing cases without available FLAIR imaging do not show values for the WMH. pHC, probability of healthy controls; pFTD, probability of FTD; pAD, probability of AD; pLBD, probability of LBD; WMH temporal, proportion of WMH in temporal lobe; WMH frontal, proportion of WMH in frontal lobe.