External validity of an automated delirium prediction model (DEMO) and comparison to the manual VMS-questions: a retrospective cohort study

Abstract

Background: It is estimated that one-third of delirium cases in hospitals could be prevented with appropriate interventions. In Dutch hospitals a manual instrument (VMS-questions) is used to identify patients at-risk for delirium. Delirium Model (DEMO) is an automated model which could support delirium prevention more efficiently. However, it has not been validated beyond the hospital it was developed in.

Aim: To externally validate the DEMO and compare its performance to the VMS-questions.

Method: A retrospective cohort study between July and December 2018 was conducted. Delirium cases were identified through a chart review, and the VMS-questions were extracted from the electronic health records. The DEMO was validated in patients ≥ 60 years, and a comparison with the VMS-questions was made in patients ≥ 70 years.

Results: In total 1,345 admissions were included. The DEMO predicted 59 out of 75 delirium cases (sensitivity 0.79, 95% CI = 0.68-0.87; specificity 0.75, 95% CI = 0.72-0.77). Compared to the VMS-questions, the DEMO showed a lower specificity (0.64 vs. 0.72; p < 0.001) and a comparable sensitivity (0.83 vs. 0.80; p = 0.56). The VMS-questions were missing in 20% of admissions, in which the DEMO correctly predicted 10 of 12 delirium cases.

Conclusion: The DEMO showed acceptable performance for delirium prediction. Overall the DEMO predicted more delirium cases because the VMS-questions were missing in 20% of admissions. This study shows that automated instruments such as DEMO could play a key role in the efficient and timely deployment of measures to prevent delirium.

Keywords: Automated model; Delirium; External validation; Hospitalized patients; Prevention measures; Risk prediction.