Artificial intelligence approach to optimise safety for hospitalised patients with dementia

Abstract

Background: The aim of the study is to develop a machine learning (ML) model to identify contributing factors to dementia-related safety events using patient safety event report data.

Method: This study uses dementia-related safety event reports from a patient safety reporting system of a 10-hospital health system in the USA. Contributing factors to safety events were coded using the Yorkshire contributory factors framework based on free-text descriptions in the reports. The coded event reports were used to develop two ML models using eXtreme Gradient Boosting (XGBoost), one to classify situational patient factors and another to classify active failures relating to human error.

Results: We used 1387 safety event reports for model development, 989 (71.3%) reports related to situational factors and 119 (8.6%) reports related to active failures. The model for situational factors achieved a precision of 0.843 and a recall of 0.826. The F1 score was 0.834, indicating a balance of precision and recall performance. The specificity of the model was 0.639 and the area under the receiver operating characteristic curve (ROC AUC) was 0.833. The final model for active failure achieved a precision of 0.333 and a recall of 0.056. The F1 score was 0.095, reflective of imbalanced precision and recall performance. The specificity of the model was 0.992, indicating a strong ability to identify negative cases, and the ROC AUC was 0.817.

Conclusion: ML techniques can provide insights into situational factors and active failures that drive dementia-related safety events. These insights can inform targeted interventions such as specialised staff training for behavioural symptoms management and pharmacist-led medication optimisation, to enhance care and safety for hospitalised people living with dementia.

Keywords: Adverse events, epidemiology and detection; Chart review methodologies; Healthcare quality improvement; Human error; Human factors.

Figure 1. Flowchart of safety events for model development. Inclusion criteria were safety events that included dementia-related keywords (dementia OR demented OR cognitively impaired OR Alzheimer’s Disease OR impaired OR vascular dementia OR Lewy body dementia OR Parkinson’s OR slow thinking OR encephalopathy OR memory loss OR altered mental status OR Frontotemporal dementia OR mild cognitive impairment). Safety events were removed if they were irrelevant (eg, if dementia was mentioned in the text but was not directly related to the safety event) or duplicates (eg, follow- up reports to a specific event), leaving a final sample of 1387 event reports which were manually coded and used in the model development.