Development and evaluation of a machine learning model predicting out-of-hospital cardiac arrest using environmental factors

Abstract

Development of an accurate system to predict the daily incidence of out-of-hospital cardiac arrest (OHCA) might provide a significant public health benefit. We developed a machine learning model to predict daily OHCA incidence at the regional level using high-resolution meteorological data, chronological data, and sociodemographic data. Among OHCAs of non-traumatic cause, 196,735 in the training dataset (2013-2017), 119,455 in the testing dataset (2018-2019) from internal areas, and 37,160 in the testing dataset from external areas were included in the analysis. Application of invariant causal prediction (ICP) successfully reduced the number of variables to 17 while maintaining high predictive performance for nationwide incidence per 100,000 per day in both the training and testing datasets, whether from internal or external regions, comparable to the non-ICP model using the initial 34 variables. Furthermore, the prediction model retained a satisfactory performance up to seven days in advance.

Fig. 1. Observed versus predicted incidence of out-of-hospital cardiac arrest by various analysis methods.

The results obtained using each method, including GAM (A), XGBoost (B), CatBoost (C), and Random Forest (D), are presented. The light blue lines indicate the observed daily incidence per 100,000 of out-of-hospital cardiac arrests in the registry participating areas. The yellow lines indicate the predicted daily incidence per 100,000 based on combined meteorological, chronological, and sociodemographic variables. GAM generalized additive model, Jan means January, XGBoost eXtreme gradient boosting.

Fig. 2. Observed versus predicted incidence of out-of-hospital cardiac arrest in the invariant causal prediction model at varying time intervals.

A shows the results of the training dataset obtained using the ICP model. B and C present the testing results for same-day, 3-day-ahead, and 7-day-ahead predictions in internal and external settings, respectively. The light blue lines indicate the observed daily incidence per 100,000 of out-of-hospital cardiac arrests in the registry-participating areas. The yellow lines indicate the predicted daily incidence per 100,000 by the XGBoost gradient boosting model using predictors selected by ICP. ICP denotes invariant causal prediction; Jan, January.

Fig. 3. Importance of variables in a machine learning prediction model.

This figure shows a variable importance plot for meteorological variables (red), chronological variables (blue), and sociodemographic variables (black) in a machine learning prediction model using XGBoost. The yellow to purple dots in each row represent low to high values for each predictor normally scaled. The x-axis shows the Shapley value, indicating the variable’s impact on the model. Positive SHAP values tend to drive predictions toward more cases of OHCA and negative SHAP values tend to drive the prediction toward fewer cases of OHCA. * In the model, 2013 was considered year 0. OHCA denotes out-of-hospital cardiac arrest; SHAP, Shapley Additive Explanations; XGBoost, eXtreme Gradient Boosting.

Fig. 4. Data sources and overview of the machine learning model for predicting out-of-hospital cardiac arrest incidence.

A illustrates the states and communities participating in the Cardiac Arrest Registry to Enhance Survival (CARES). B displays an example of daily maximum ambient temperature data on July 15, 2022, obtained from the North American Land Data Assimilation System (NLDAS). C summarizes the development process of the machine learning model used to predict out-of-hospital cardiac arrest incidence.