Development and validation of the Florey Dementia Risk Score web-based tool to screen for Alzheimer's disease in primary care

Abstract

Background: It is estimated that ∼60% of people with Alzheimer's disease (AD) are undetected or undiagnosed, with higher rates of underdiagnosis in low-to middle-income areas with limited medical resources. To promote health equity, we have developed a web-based tool that utilizes easy-to-collect clinical data to enhance AD detection rate in primary care settings.

Methods: This study was leveraged on the data collected from participants of the Australian Imaging, Biomarker & Lifestyle (AIBL) study and the Religious Orders Study and Memory and Aging Project (ROSMAP). The study included three phases: (1) constructing and evaluating a model on retrospective cohort data (1407 AIBL participants), (2) performing simulated trials to assess model accuracy (30 AIBL participants) and missing data tolerability (30 AIBL participants), and (3) external evaluation using a non-Australian dataset (500 ROSMAP participants). The auto-score machine learning algorithm was employed to develop the Florey Dementia Risk Score (FDRS). All the simulated trials and evaluation were performed using a web-based FDRS tool.

Findings: FDRS achieved an area under the curve (AUC) of approximately 0.82 [95% CI, 0.75-0.88], with a sensitivity of 0.74 [0.60-0.86] and a specificity of 0.73 [0.70-0.79]. The accuracy of the simulated pilot trial for 30 AIBL participants with complete record was 87% (26/30 correct), while it only slightly decreased (80.0-83.3%, depending on imputation methods) for another 30 AIBL participants with one or two missing data. FDRS achieved an AUC of 0.82 [0.77-0.86] of 500 ROSMAP participants.

Interpretation: The FDRS tool offers a potential low-cost solution to AD screening in primary care. The present study warrants future trials of FDRS for optimization and to confirm its generalizability across a more diverse population, especially people in low-income countries.

Funding: National Health and Medical Research Council, Australia (GNT2007912) and Alzheimer's Association, USA (23AARF-1020292).

Keywords: Alzheimer's disease; Auto-score algorithm; Binary classification; Disease screening; Health equity; Web-based tool.

Fig. 1

Importance ranking of features. The x-axis displays the importance determined by the random forest feature selection, with a higher numerical value indicating higher importance, while the y-axis lists the name of 35 features collected in the Australian Imaging, Biomarker & Lifestyle study. The plot illustrates the relative importance of various features. Age is the most important feature, while epilepsy is the least important one. Abbreviations: Geriatric Depression Scale (GDS), apolipoprotein E (APOE).

Fig. 2

Parsimony plot for the Florey Dementia Risk Score (FDRS) model using a cumulative number of features. This plot was obtained using the training set, which shows the area under curve (AUC) values when increasing number of features are used in the FDRS model. The number within the bar represents the total number of features used, and the height of the bar indicates the mean AUC value from a 10-fold cross validation. For example, the third bar on left means when the first three features are used for FDRS, the AUC is ∼0.7 as indicated on the y-axis. Abbreviations: Geriatric Depression Scale (GDS), apolipoprotein E (APOE).

Fig. 3

Receiver operating characteristic (ROC) plot on the Religious Orders Study and Memory and Aging Project dataset. The ROC curve illustrates the trade-off between sensitivity (true positive rate) and specificity (1–false positive rate) for different threshold settings. The blue line is the ROC curve when the cutoff score is 66 for Alzheimer's disease binary classification. The red dashed line represents random guessing for Alzheimer's disease binary classification.

Fig. 4

Histogram of frequency distribution of Florey Dementia Risk Score (FDRS) for participants in Religious Orders Study and Memory and Aging Project (ROSMAP). The plot shows the distribution of FDRS scores of 500 ROSMAP study participants, with a cutoff value of 66 indicated by the red dashed line separating cognitive unimpaired (CU, left) and Alzheimer's disease (AD, right). The clinical diagnoses of the participants are color-coded, with green for CU and yellow for AD. It appears that most of the CU and AD subjects can be correctly classified.