Development and validation of a novel predictive model for dementia risk in middle-aged and elderly depression individuals: a large and longitudinal machine learning cohort study

Abstract

Background: Depression serves as a prodromal symptom of dementia, and individuals with depression exhibit a significantly higher risk of developing dementia. The aim of this study is to develop and validate a novel dementia risk prediction tool among middle-aged and elderly individuals with depression based on machine learning algorithms.

Methods: This study included 31,587 middle-aged and elderly individuals with depression who did not have a diagnosis of dementia at baseline from a large UK population-based prospective cohort. A rigorous variable selection strategy was employed to identify risk and protective factors of dementia from an initial pool of 190 candidate variables, ultimately retaining 27 variables. Eight distinct data analysis strategies were utilized to develop and validate the dementia risk prediction model. The DeLong's test was applied to compare the statistical differences between different models.

Results: During a median follow-up of 7.98 years, 896 incident dementia cases were identified among study participants. In model development employing an 8:2 data split (fivefold cross-validation for training), the Adaboost classifier achieved the optimal performance (AUC 0.861 ± 0.003), followed by XGBoost (AUC 0.839 ± 0.005) and CatBoost (AUC 0.828 ± 0.007) classifiers. To facilitate community generalization and clinical applicability, we develop a simplified model through a forward feature subset selection algorithm, retaining 12 variables. The simplified model maintained robust performance, with AdaBoost achieving the highest discriminative ability (AUC 0.859 ± 0.002), followed by XGBoost (AUC 0.835 ± 0.001) and CatBoost (AUC 0.821 ± 0.005). The DeLong's test revealed no statistically significant difference in AUC values between models using 12 and 27 variables (p = 0.278). For practical implementation, we deployed the optimal model to a web application for visualization and dementia risk assessment, named DRP-Depression.

Conclusions: We developed a practical and easy-to-promote risk prediction model based on machine learning algorithms, and deployed it to a web application to provide a new and convenient tool for dementia risk prediction in the middle-aged and elderly individuals with depression.

Keywords: DRP-Depression; Dementia; Depression; Machine learning; Risk prediction; UK Biobank; Web application.

Fig. 1

Experimental flow chart. The panels exhibit the development pipeline of the machine learning model. The diagram shows the key steps of model development, as well as sample diagrams of performance evaluation and web application process

Fig. 2

Model performance, feature selection results, and model interpretation. Area under the receiver operating characteristic curve (AUC) plots of different classifiers of 27 variables in Study 4. A The line chart depicts the change in AUC during the selection of a subset of features. 12 variables were finally remained for model optimization. B Area under the receiver operating characteristic curve (AUC) plots of different classifiers of 12 variables in Study 4. C The bar chart represents the sorted 12 variables based on their importance to the AdaBoost in Study 4. D SHAP visualization plot of the selected 12-variable simplified model. The specific effect of each variable on the model can be interpreted by its value magnitude (encoded by the color gradient) and tendency direction (on the horizontal axis) for the SHAP plot, where each datapoint represents an individual case's feature contribution in each row, with superimposed violin plots illustrating population-level effect size distributions (a red point represents a large feature value, and a blue point represents a small feature value). The horizontal displacement indicates effect direction (rightward: risk elevation; leftward: protective effect). When the SHAP value is greater than zero (right side), a larger value magnitude represents a higher risk of developing dementia. In contrast, when the SHAP value is less than zero (left side), a smaller value magnitude represents a larger likelihood against developing dementia. Taking age as an example, older participants (colored in red) were more likely to develop dementia (right side), and younger participants (colored in blue) had a larger likelihood against developing dementia (left side)

Fig. 3

Model performance comparison charts, robustness analysis line charts, and Webpage interface of predictive model. A Comparison of models performance between 27 variables and 12 variables in Study 4 using the DeLong's test, and p < 0.05 suggested that there was a significant difference. B Comparison of the 3 classifiers (27 variables) in Study 4 using the DeLong's test, and p < 0.05 suggested that there was a significant difference. C Comparison of the 3 classifiers (12 variables) in Study 4 using the DeLong's test, and p < 0.05 suggested that there was a significant difference. D The line chart shows the robustness of the model after 50 random seed selections in Study 4. E The baseline characteristics can be entered on the left operation bar, and the right display bar shows the dementia risk in a pie chart and a dashboard form and provides reasonable prevention suggestions based on the baseline situation