An interpretable machine learning tool for in-home monitoring of agitation episodes in people living with dementia: a proof-of-concept study

Abstract

Background: Agitation affects around 30% of people living with dementia (PLwD), increasing carer burden and straining care services. Agitation identification typically relies on subjective clinical scales and direct patient observation, which are resource-intensive and challenging to incorporate into routine care. Clinical applicability of data-driven methods for agitation monitoring is limited by constraints such as short observational periods, data granularity, and lack of interpretability and generalisation. Current interventions for agitation are primarily medication-based, which may lead to severe side effects and lack personalisation. Understanding how real-world factors interact with agitation within home settings offers a promising avenue towards identifying potential personalised non-pharmacological interventions.

Methods: We used longitudinal data (32,896 person-days from n = 63 PLwD) collected using in-home monitoring devices between December 2020 and March 2023. Employing machine learning techniques, we developed a monitoring tool to identify the presence of agitation during the week. We incorporated a traffic-light system to stratify agitation probability estimates supporting clinical decision-making, and employed the SHapley Additive exPlanations (SHAP) framework to enhance interpretability. We designed an interactive tool that enables the exploration of personalised non-pharmacological interventions, such as modifying ambient light and temperature.

Findings: Light Gradient-boosting Machine (LightGBM) achieved the highest performance in identifying agitation over an 8-day period with a sensitivity of 71.32% ± 7.38 and specificity of 75.28% ± 7.38. Implementing the traffic-light system for stratification increased specificity to 90.3% ± 7.55 and improved all metrics. Key features for identifying agitation included low nocturnal respiratory rate, heightened alertness during sleep, and increased indoor illuminance, as revealed by statistical and feature importance analysis. Using our interactive tool, we identified indoor lighting and temperature adjustments as the most promising and feasible intervention options within our cohort.

Interpretation: Our interpretable framework for agitation monitoring, developed using data from a dementia care study, showcases significant clinical value. The accompanying interactive interface allows for the in-silico simulation of non-pharmacological interventions, facilitating the design of personalised interventions that can improve in-home dementia care.

Funding: This study is funded by the UK Dementia Research Institute [award number UK DRI-7002] through UK DRI Ltd, principally funded by the Medical Research Council (MRC), and the UKRI Engineering and Physical Sciences Research Council (EPSRC) PROTECT Project (grant number: EP/W031892/1). Infrastructure support for this research was provided by the NIHR Imperial Biomedical Research Centre (BRC) and the UKRI Medical Research Council (MRC). P.B. is also funded by the Great Ormond Street Hospital and the Royal Academy of Engineering. C.S. is supported by the UK Dementia Research Institute [award number UK DRI-5209], a UKRI Future Leaders Fellowship [MR/MR/X032892/1] and the Edmond J. Safra Foundation. R.N. is funded by UK Dementia Research Institute [award number UK DRI-7002] and the UKRI Engineering and Physical Sciences Research Council (EPSRC) PROTECT Project (grant number: EP/W031892/1). M.B. and A.K.S. are funded by the UK Dementia Research Institute [award number UKDRI-7002 and UKDRI-5209]. N.F.L., A.C., C.W. and S.K. are funded by the UK Dementia Research Institute [award number UK DRI-7002].

Keywords: Agitation; Dementia care; Digital health tools; Machine learning; Remote monitoring.

Fig. 1

Overview of approach. The data processing and analysis pipeline are shown, illustrating the specific steps followed. Our two strategies for enhanced clinical value are showcased: an interactive interface for personalised in-silico intervention experiments and a traffic-light system to minimise false alerts. Figure was created with Biorender.

Fig. 2

Model interpretability through feature importance analysis. The feature importance calculated using SHapley Additive exPlanations (SHAP) on the test sets from the 10-Fold cross-validation is shown in a summary plot. The colour represents the scaled feature value (red corresponding to higher values, blue to lower). The position of the x-axis represents the contribution of each normalised feature value to the positive prediction of agitation.

Fig. 3

Personalised investigation of modifiable features. Examples from a week with presence of agitation (a) and a week without agitation (b) for PLwD A are shown using the SHapley Additive exPlanations (SHAP) framework. The colour of the arrow corresponds to the contribution: red contributes to agitation presence and blue contributes to agitation absence. Positive SHAP values contributed to positive predictions (agitated), while negative SHAP values contributed to negative predictions (non-agitated). The size of the arrow represents the absolute SHAP value, indicating the magnitude of each feature's contribution. The number within the arrow corresponds to the normalised feature value.

Fig. 4

In-silico experiment: Adjusting Temperature via an Interactive Interface. The interactive interface is shown, which accepts the input data as a CSV file. The tool provides sliding bars for the modifiable features and presents the associated probability estimates, and a feature importance plot using SHapley Additive exPlanations (SHAP) values. In the feature importance plot, red bars correspond to features that contributed to the model's decision-making towards agitation and blue bars correspond to features that contribute towards absence of agitation. Each bar is annotated with the corresponding normalised feature value. The user can save the combinations of modifications they have made to the modifiable parameters. a. Anonymised data from a participant. b. The results after modifying one of the parameters, evening indoor bathroom temperature. An online version with a synthetic patient data generator is hosted on huggingface (see https://huggingface.co/spaces/marirena/AgitationMonitoring).