Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2022 Mar;70(3):831-837.
doi: 10.1111/jgs.17491. Epub 2021 Oct 13.

Machine learning-assisted screening for cognitive impairment in the emergency department

Simon R Yadgir  1 , Collin Engstrom  1   2 , Gwen Costa Jacobsohn  1 , Rebecca K Green  1 , Courtney M C Jones  3   4 , Jeremy T Cushman  3   4   5 , Thomas V Caprio  6 , Amy J H Kind  7   8   9 , Michael Lohmeier  1 , Manish N Shah  1   7   10 , Brian W Patterson  1   11   12
Affiliations
Randomized Controlled Trial

Machine learning-assisted screening for cognitive impairment in the emergency department

Simon R Yadgir et al. J Am Geriatr Soc. 2022 Mar.

Abstract

Background/objectives: Despite a high prevalence and association with poor outcomes, screening to identify cognitive impairment (CI) in the emergency department (ED) is uncommon. Identification of high-risk subsets of older adults is a critical challenge to expanding screening programs. We developed and evaluated an automated screening tool to identify a subset of patients at high risk for CI.

Methods: In this secondary analysis of existing data collected for a randomized control trial, we developed machine-learning models to identify patients at higher risk of CI using only variables available in electronic health record (EHR). We used records from 1736 community-dwelling adults age > 59 being discharged from three EDs. Potential CI was determined based on the Blessed Orientation Memory Concentration (BOMC) test, administered in the ED. A nested cross-validation framework was used to evaluate machine-learning algorithms, comparing area under the receiver-operator curve (AUC) as the primary metric of performance.

Results: Based on BOMC scores, 121 of 1736 (7%) participants screened positive for potential CI at the time of their ED visit. The best performing algorithm, an XGBoost model, predicted BOMC positivity with an AUC of 0.72. With a classification threshold of 0.4, this model had a sensitivity of 0.73, a specificity of 0.64, a negative predictive value of 0.97, and a positive predictive value of 0.13. In a hypothetical ED with 200 older adult visits per week, the use of this model would lead to a decrease in the in-person screening burden from 200 to 77 individuals in order to detect 10 of 14 patients who would fail a BOMC.

Conclusion: This study demonstrates that an algorithm based on EHR data can define a subset of patients at higher risk for CI. Incorporating such an algorithm into a screening workflow could allow screening efforts and resources to be focused where they have the most impact.

Keywords: cognitive impairment; delirium; dementia; emergency medicine; machine learning.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest

The authors have no conflicts.

Figures

Figure 1.
Figure 1.
Model training framework flow diagram. AUC: Area under the curve.
See this image and copyright information in PMC

Comment in

References

    1. LaMantia M, Stump TE, Messina FC, Miller DK,Callahan CM. Emergency Department Use Among Older Adults With Dementia. Alzheimer Dis Assoc Disord 2016;30(1):35–40. DOI: 10.1097/WAD.0000000000000118. - DOI - PMC - PubMed
    1. Han JH, Bryce SN, Ely EW et al. The effect of cognitive impairment on the accuracy of the presenting complaint and discharge instruction comprehension in older emergency department patients. Ann Emerg Med 2011;57(6):662–671.e2. DOI: 10.1016/j.annemergmed.2010.12.002. - DOI - PMC - PubMed
    1. Hustey FM, Meldon SW. The Prevalence and Documentation of Impaired Mental Status in Elderly Emergency Department Patients. Ann Emerg Med 2002;39(3):248–253. DOI: 10.1067/mem.2002.122057. - DOI - PubMed
    1. Han JH, Suyama J. Delirium and Dementia. Clin Geriatr Med 2018;34(3):327–354. DOI: 10.1016/j.cger.2018.05.001. - DOI - PubMed
    1. Kakuma R, Galbaud du Fort G, Arsenault L et al. Delirium in Older Emergency Department Patients Discharged Home: Effect on Survival. J Am Geriatr Soc 2003;51(4):443–450. DOI: 10.1046/j.1532-5415.2003.51151.x. - DOI - PubMed

Publication types