A Novel, Interpretable Machine Learning Model to Predict Neurological Outcomes Following Venoarterial Extracorporeal Membrane Oxygenation

Abstract

Background: We used machine learning models incorporating rich electronic medical record (EMR) data to predict neurological outcomes after venoarterial extracorporeal membrane oxygenation (VA-ECMO).

Methods: This was a retrospective review of adult (≥ 18 years) patients undergoing VA-ECMO between 6/2016 and 4/2022 at a single center. The primary outcome was good neurological outcome, defined as a modified Rankin Scale score of 0 to 3, evaluated at hospital discharge. We extracted every measurement of 74 vital and laboratory values, as well as circuit and ventilator settings, from 24 h before cannulation through the entire duration of ECMO. An XGBoost model with Shapley Additive Explanations was developed and evaluated with leave-one-out cross-validation.

Results: Overall, 194 patients undergoing VA-ECMO (median age 58 years, 63% male) were included. We extracted more than 14 million individual data points from the EMR. Of 194 patients, 39 patients (20%) had good neurological outcomes. Three models were generated: model A, which contained only pre-ECMO data; model B, which added data from the first 48 h of ECMO; and model C, which included data from the entire ECMO run. The leave-one-out cross-validation area under the receiver operator characteristics curves for models A, B, and C were 0.72, 0.81, and 0.90, respectively. The inclusion of on-ECMO physiologic, laboratory, and circuit data greatly improved model performance. Both modifiable and nonmodifiable variables, such as lower body mass index, lower age, higher mean arterial pressure, and higher hemoglobin, were associated with good neurological outcome.

Conclusions: An interpretable machine learning model from EMR-extracted data was able to predict neurological outcomes for patients undergoing VA-ECMO with excellent accuracy.

Keywords: ECMO; Machine learning; Neurological outcomes.

Fig. 1

Patient cohort selection. A total of 310 adult patients underwent ECMO support between June 2016 and April 2022. Patients with VV-ECMO and ECPR (n = 102) were excluded, and an additional 14 patients undergoing VA-ECMO without available laboratory data or determination of neurological outcome were excluded. The final cohort was 194 patients undergoing VA-ECMO, of whom 39 (20%) had a good neurological outcome. ECMO, extracorporeal membrane oxygenation, ECPR, extracorporeal cardiopulmonary resuscitation, mRS, modified Rankin Scale, VV, venovenous, VA, venoarterial

Fig. 2

Variables included in model training. Electronic medical record query and chart review yielded 14 million individual measurements available for analysis. BMI, body mass index, BP, blood pressure, BUN, blood urea nitrogen, CAM-ICU, Confusion Assessment Method-Intensive Care Unit, ECMO, extracorporeal membrane oxygenation, MCH, mean corpuscular hemoglobin, MCHC, mean corpuscular hemoglobin concentration, MCV, mean corpuscular volume, RASS, Richmond Agitation-Sedation Scale, RBC, red blood cell, RDW, red blood cell distribution width, WBC, white blood cell

Fig. 3

Model performance. Receiver operator characteristics curves for leave-one-out-cross-validation models A (green line, only pre-ECMO features), B (orange line, with data from the first 48 h of ECMO support), and C (blue line, with data from the entire ECMO run). AUC, area under the curve, ECMO, extracorporeal membrane oxygenation

Fig. 4

Most important features for each model. Ranked Shapley values for the 20 variables in each machine learning model that provide the most discriminative value. Each dot is a patient, with red and blue values representing higher and lower numerical values, respectively, of the listed variable. Dots to the right and left of the vertical line indicate that the variable was associated with a model prediction of “good” or “poor” neurological outcome, respectively. Variables are vertically ranked by their relative importance to model predictions, calculated by the mean absolute value of the Shapley values. As such, variables with the largest spread along the x-axis are ranked higher, indicating their higher importance in distinguishing between good and poor neurologic outcomes. BP, blood pressure, ECMO, extracorporeal membrane oxygenation, IABP, intraaortic balloon pump, MAP, mean arterial pressure, pRBC, packed red blood cell, RBC, red blood cell, TV, tidal volume

Fig. 5

Shapley force plots for two individual example patients. Features in red drove the model to predict a good neurological outcome (as illustrated by arrows on the line graph pointing in the positive direction) while features in blue drove the prediction toward a poor outcome. The “f(x)” value denotes the strength of the final prediction, with negative and positive values representing a predicted poor and good outcome, respectively. Please note that the x-axis scales are different between the two presented patients. BMI, body mass index, Max, maximum, pRBC, packed red blood cell, SpO₂, oxygen saturation