Alzheimer's Disease Assessments Optimized for Diagnostic Accuracy and Administration Time

Abstract

Objective: Despite the potential of machine learning techniques to improve dementia diagnostic processes, research outcomes are often not readily translated to or adopted in clinical practice. Importantly, the time taken to administer diagnostic assessment has yet to be taken into account in feature-selection based optimisation for dementia diagnosis. We address these issues by considering the impact of assessment time as a practical constraint for feature selection of cognitive and functional assessments in Alzheimer's disease diagnosis.

Methods: We use three different feature selection algorithms to select informative subsets of dementia assessment items from a large open-source dementia dataset. We use cost-sensitive feature selection to optimise our feature selection results for assessment time as well as diagnostic accuracy. To encourage clinical adoption and further evaluation of our proposed accuracy-vs-cost optimisation algorithms, we also implement a sandbox-like toolbox with graphical user interface to evaluate user-chosen subsets of assessment items.

Results: We find that there are subsets of accuracy-cost optimised assessment items that can perform better in terms of diagnostic accuracy and/or total assessment time than most other standard assessments.

Discussion: Overall, our analysis and accompanying sandbox tool can facilitate clinical users and other stakeholders to apply their own domain knowledge to analyse and decide which dementia diagnostic assessment items are useful, and aid the redesigning of dementia diagnostic assessments. Clinical Impact (Clinical Research): By optimising diagnostic accuracy and assessment time, we redesign predictive and efficient dementia diagnostic assessments and develop a sandbox interface to facilitate evaluation and testing by clinicians and non-specialists.

Keywords: Cost-sensitive feature selection; assessment speed-accuracy trade-off; cognitive and functional assessments; dementia and Alzheimer’s disease diagnosis; sandbox GUI application.

FIGURE 1.

Pre-processing and analytical pipeline for selecting accuracy-optimised sets of dementia assessment items. Note: Step 2 shows an instance of feature selection - performed on Fold 3.

FIGURE 2.

Analytical pipeline for optimising accuracy-cost dementia assessment items.

FIGURE 3.

The most consistently selected CFA items. Only data features selected more than once in the iterations feature selection process are shown. Bold text: Features consistently selected 10 or more times.

FIGURE 4.

Multiclass AUC of each selected feature set. Dashed line: AUC using all features.

FIGURE 5.

Wide range of (estimated) total assessment times of accuracy-cost optimised feature sets. Vertical axis: AUC values based on 3-class RF classifier. Horizontal axis: Logarithmic scale of (estimated) total assessment time. AUCs of accuracy-cost optimised sets (filled circles) higher than ADAS, MMSE and MoCA complete standardised assessments while on par with FAQ complete assessment (black opened circles). Orange filled circle: Optimal item set with the shortest total assessment time (see Table 1 for details on its specific items).

FIGURE 6.

Different features of the cost-benefit analytical sandbox tool. (a)-(e): Order of features appearing during usage.