Development and Validation of a Machine Learning Model to Predict Near-Term Risk of Iatrogenic Hypoglycemia in Hospitalized Patients

Abstract

Importance: Accurate clinical decision support tools are needed to identify patients at risk for iatrogenic hypoglycemia, a potentially serious adverse event, throughout hospitalization.

Objective: To predict the risk of iatrogenic hypoglycemia within 24 hours after each blood glucose (BG) measurement during hospitalization using a machine learning model.

Design, setting, and participants: This retrospective cohort study, conducted at 5 hospitals within the Johns Hopkins Health System, included 54 978 admissions of 35 147 inpatients who had at least 4 BG measurements and received at least 1 U of insulin during hospitalization between December 1, 2014, and July 31, 2018. Data from the largest hospital were split into a 70% training set and 30% test set. A stochastic gradient boosting machine learning model was developed using the training set and validated on internal and external validation.

Exposures: A total of 43 clinical predictors of iatrogenic hypoglycemia were extracted from the electronic medical record, including demographic characteristics, diagnoses, procedures, laboratory data, medications, orders, anthropomorphometric data, and vital signs.

Main outcomes and measures: Iatrogenic hypoglycemia was defined as a BG measurement less than or equal to 70 mg/dL occurring within the pharmacologic duration of action of administered insulin, sulfonylurea, or meglitinide.

Results: This cohort study included 54 978 admissions (35 147 inpatients; median [interquartile range] age, 66.0 [56.0-75.0] years; 27 781 [50.5%] male; 30 429 [55.3%] White) from 5 hospitals. Of 1 612 425 index BG measurements, 50 354 (3.1%) were followed by iatrogenic hypoglycemia in the subsequent 24 hours. On internal validation, the model achieved a C statistic of 0.90 (95% CI, 0.89-0.90), a positive predictive value of 0.09 (95% CI, 0.08-0.09), a positive likelihood ratio of 4.67 (95% CI, 4.59-4.74), a negative predictive value of 1.00 (95% CI, 1.00-1.00), and a negative likelihood ratio of 0.22 (95% CI, 0.21-0.23). On external validation, the model achieved C statistics ranging from 0.86 to 0.88, positive predictive values ranging from 0.12 to 0.13, negative predictive values of 0.99, positive likelihood ratios ranging from 3.09 to 3.89, and negative likelihood ratios ranging from 0.23 to 0.25. Basal insulin dose, coefficient of variation of BG, and previous hypoglycemic episodes were the strongest predictors.

Conclusions and relevance: These findings suggest that iatrogenic hypoglycemia can be predicted in a short-term prediction horizon after each BG measurement during hospitalization. Further studies are needed to translate this model into a real-time informatics alert and evaluate its effectiveness in reducing the incidence of inpatient iatrogenic hypoglycemia.

Figure 1.. Study Flowchart

All patients had at least 4 blood glucose (BG) measurements, received at least 1 U of subcutaneous insulin during admission, and were admitted to the hospital between December 1, 2014, and July 31, 2018. ICU indicates intensive care unit; IV, intravenous; and LOS, length of stay.

Figure 2.. Variable Importance Plot of Top 30 Predictor Variables

BG indicates blood glucose; BMI, body mass index; CV, coefficient of variation; eGFR, estimated glomerular filtration rate; WBC, white blood cell count.

Figure 3.. Prediction Horizon for a Sample Patient in the Data Set, Showing Only 7 of the 43 Predictor Variables in the Model

The prediction model uses data of all predictor variables during hospital stay up to the time of index blood glucose (BG) value to predict outcome in the 24 hours after the index BG value. Weight is an example of a time-fixed variable, whereas all other variables shown are time varying. Orange and blue arrows indicate 24-hour prediction horizons from time of index BG values in which iatrogenic hypoglycemia is absent and present, respectively. bpm indicates beats per minute; CV, coefficient of variation; eGFR, estimated glomerular filtration rate.