Development and validation of a machine learning model for classification of next glucose measurement in hospitalized patients

Abstract

Background: Inpatient glucose management can be challenging due to evolving factors that influence a patient's blood glucose (BG) throughout hospital admission. The purpose of our study was to predict the category of a patient's next BG measurement based on electronic medical record (EMR) data.

Methods: EMR data from 184,361 admissions containing 4,538,418 BG measurements from five hospitals in the Johns Hopkins Health System were collected from patients who were discharged between January 1, 2015 and May 31, 2019. Index BGs used for prediction included the 5th to penultimate BG measurements (N = 2,740,539). The outcome was category of next BG measurement: hypoglycemic (BG $\le$ 70 mg/dl), controlled (BG 71-180 mg/dl), or hyperglycemic (BG > 180 mg/dl). A random forest algorithm that included a broad range of clinical covariates predicted the outcome and was validated internally and externally.

Findings: In our internal validation test set, 72·8%, 25·7%, and 1·5% of BG measurements occurring after the index BG were controlled, hyperglycemic, and hypoglycemic respectively. The sensitivity/specificity for prediction of controlled, hyperglycemic, and hypoglycemic were 0·77/0·81, 0·77/0·89, and 0·73/0·91, respectively. On external validation in four hospitals, the ranges of sensitivity/specificity for prediction of controlled, hyperglycemic, and hypoglycemic were 0·64-0·70/0·80-0·87, 0·75-0·80/0·82-0·84, and 0·76-0·78/0·87-0·90, respectively.

Interpretation: A machine learning algorithm using EMR data can accurately predict the category of a hospitalized patient's next BG measurement. Further studies should determine the effectiveness of integration of this model into the EMR in reducing rates of hypoglycemia and hyperglycemia.

Keywords: AUC, area under receiver operating curve; BG, blood glucose; BMI, body mass index; CGM, continuous glucose monitor; EMR, electronic medical record; ICD, International Classification of Diseases; ICU, intensive care unit; NLR, negative likelihood ratio; NPO, nil per os; NPV, negative predictive value; PLR, positive likelihood ratio; PPV, positive predictive value; T1DM, type 1 diabetes mellitus; T2DM, type 2 diabetes mellitus.

Figure 1

Study flowchart. *These BG readings were included as historical glucose data in the admission, but not as index observations.

Figure 2

Lookback window, index observation, and prediction horizon. Top: Starting with the 5th blood glucose (BG) measurement, our machine learning algorithm begins to make predictions about a patient's next BG measurement in an expected prediction horizon of five minutes-ten hours. Data preceding the 5th BG measurement are included in summary statistics over the previous 24 h and admission as applicable. Bottom: As a patient's admission becomes longer, BG measurements that were earlier in the admission may no longer be included in the 24 h summary statistics but will continue to be included in the admission summary statistics. The five minute to ten hour prediction horizon continues to roll with each new BG measurement.

Figure 3

Time to next blood glucose reading in test set of Hospital 1. Red lines mark the middle 90% (5th percentile-95th percentile) and blue lines mark the middle 50% (25th percentile–75th percentile).

Figure 4

Variable Importance Plot of Top 20 Predictors. Variable importance plot based on the mean decrease in Gini, which is a measure of how much heterogeneity (i.e. misclassification) is lost when a predictor is used in a random forest node.

Figure 5

Probability cutpoints to determine the predicted class of a patient's next BG reading. This shows how probabilities for each BG category are used in an algorithmic fashion to determine the final class of BG. Cutpoints were selected that maximized the sum of sensitivity and specificity for each class.

Figure 6

AUC performance of each class of prediction by hospital. AUC curves were plotted for each class (controlled, hyperglycemia, and hypoglycemia) by comparing the sensitivity and specificity at different cutpoints for each class individually (controlled vs. not controlled, hyperglycemic vs. not hyperglycemic, and hypoglycemic vs. not hypoglycemic). The AUC curves of Hospital 1 are the class-specific model performance in the test set and the AUC curves of Hospitals 2–5 are the class-specific model performances in the external validation sets.