Assessment of Machine Learning to Estimate the Individual Treatment Effect of Corticosteroids in Septic Shock

Abstract

Importance: The survival benefit of corticosteroids in septic shock remains uncertain.

Objective: To estimate the individual treatment effect (ITE) of corticosteroids in adults with septic shock in intensive care units using machine learning and to evaluate the net benefit of corticosteroids when the decision to treat is based on the individual estimated absolute treatment effect.

Design, setting, and participants: This cohort study used individual patient data from 4 trials on steroid supplementation in adults with septic shock as a training cohort to model the ITE using an ensemble machine learning approach. Data from a double-blinded, placebo-controlled randomized clinical trial comparing hydrocortisone with placebo were used for external validation. Data analysis was conducted from September 2019 to February 2020.

Exposures: Intravenous hydrocortisone 50 mg dose every 6 hours for 5 to 7 days with or without enteral 50 μg of fludrocortisone daily for 7 days. The control was either the placebo or usual care.

Main outcomes and measures: All-cause 90-day mortality.

Results: A total of 2548 participants were included in the development cohort, with median (interquartile range [IQR]) age of 66 (55-76) years and 1656 (65.0%) men. The median (IQR) Simplified Acute Physiology Score (SAPS II) was 55 [42-69], and median (IQR) Sepsis-related Organ Failure Assessment score on day 1 was 11 (9-13). The crude pooled relative risk (RR) of death at 90 days was 0.89 (95% CI, 0.83 to 0.96) in favor of corticosteroids. According to the optimal individual model, the estimated median absolute risk reduction was of 2.90% (95% CI, 2.79% to 3.01%). In the external validation cohort of 75 patients, the area under the curve of the optimal individual model was 0.77 (95% CI, 0.59 to 0.92). For any number willing to treat (NWT; defined as the acceptable number of people to treat to avoid 1 additional outcome considering the risk of harm associated with the treatment) less than 25, the net benefit of treating all patients vs treating nobody was negative. When the NWT was 25, the net benefit was 0.01 for the treat all with hydrocortisone strategy, -0.01 for treat all with hydrocortisone and fludrocortisone strategy, 0.06 for the treat by SAPS II strategy, and 0.31 for the treat by optimal individual model strategy. The net benefit of the SAPS II and the optimal individual model treatment strategies converged to zero for a smaller number willing to treat, but the individual model was consistently superior than model based on the SAPS II score.

Conclusions and relevance: These findings suggest that an individualized treatment strategy to decide which patient with septic shock to treat with corticosteroids yielded positive net benefit regardless of potential corticosteroid-associated side effects.

Figure 1.. Model Discrimination

AUC indicates area under the receiver operating characteristic curve; SAPS II, Simplified Acute Physiology Score.

Figure 2.. Optimal Individual Model Calibration

Calibration plot using 10-fold cross-validation. The dotted line indicates perfect calibration; blue line, calibration obtained with the Super Learner; and shading, 95% CI.

Figure 3.. Expected Net Benefit Based on the Number Willing to Treat (NWT)

The y-axis is the net benefit for each treatment strategy compared with treating no one. Treating no one served as a reference and is equal to zero. For treat all patients and treat based on the Simplified Acute Physiology Score (SAPS II), the treatment considered is either hydrocortisone alone or hydrocortisone with fludrocortisone. For the optimal individual model, the treatment is the one expected to produce the maximal effect at the individual level. The x-axis is the NWT, which is equal to 1 / decision threshold. Shading indicates 95% CI.