Unexplained mortality differences between septic shock trials: a systematic analysis of population characteristics and control-group mortality rates

Abstract

Purpose: Although the definition of septic shock has been standardized, some variation in mortality rates among clinical trials is expected. Insights into the sources of heterogeneity may influence the design and interpretation of septic shock studies. We set out to identify inclusion criteria and baseline characteristics associated with between-trial differences in control group mortality rates.

Methods: We conducted a systematic review of RCTs published between 2006 and 2018 that included patients with septic shock. The percentage of variance in control-group mortality attributable to study heterogeneity rather than chance was measured by I². The association between control-group mortality and population characteristics was estimated using linear mixed models and a recursive partitioning algorithm.

Results: Sixty-five septic shock RCTs were included. Overall control-group mortality was 38.6%, with significant heterogeneity (I² = 93%, P < 0.0001) and a 95% prediction interval of 13.5-71.7%. The mean mortality rate did not differ between trials with different definitions of hypotension, infection or vasopressor or mechanical ventilation inclusion criteria. Population characteristics univariately associated with mortality rates were mean Sequential Organ Failure Assessment score (standardized regression coefficient (β) = 0.57, P = 0.007), mean serum creatinine (β = 0.48, P = 0.007), the proportion of patients on mechanical ventilation (β = 0.61, P < 0.001), and the proportion with vasopressors (β = 0.57, P = 0.002). Combinations of population characteristics selected with a linear model and recursive partitioning explained 41 and 42%, respectively, of the heterogeneity in mortality rates.

Conclusions: Among 65 septic shock trials, there was a clinically relevant amount of heterogeneity in control group mortality rates which was explained only partly by differences in inclusion criteria and reported baseline characteristics.

Keywords: Clinical trials; Heterogeneity; Machine learning; Meta-research; Methodology; Septic shock.

Fig. 1

Control-group mortality rates categorized by trial inclusion criteria. The diamonds represent the mean mortality rates and 95% confidence intervals. The 95% prediction intervals (dashed lines) represents the estimated between-trial variability in mortality rates after adjusting for random chance and sample size. I² represents the proportion of between-trial variability that cannot be explained by chance. There were no significant differences in mean mortality rates between inclusion criteria. MAP mean arterial pressure, SBP systolic blood pressure

Fig. 2

Heatmap of included trials (n = 65) and associated baseline characteristics, ranked by decreasing mortality rates. White tiles represent the mean value across trials, while red and blue tiles are indicative of higher and lower than average values, respectively. Gray tiles (N/A) are variables that were not reported. The 28-day mortality rate ranged between 13.8 and 84.6%, with a mean of 38.6%. APACHE Acute Physiology and Chronic Health Evaluation, SAPS Simplified Acute Physiology Score, SOFA Sequential Organ Failure Assessment score, MAP mean arterial pressure, CVP central venous pressure, CNS central nervous system. (Asterisk) Variables with a significant univariate association with 28-day mortality

Fig. 3

Included trials ordered by predicted control group mortality rate (diamonds). The predicted mortality rates were based on a multivariate weighted random-effects regression model with baseline mean Sequential Organ Failure Assessment (SOFA) score, the proportion of patients on mechanical ventilation, and mean serum creatinine as significant independent variables. The squares and brackets are the observed control-group mortality rates with 95% confidence interval. The figure illustrates that the model explained (R²) 41% of the variability in mortality rates, with significant residual heterogeneity (P < 0.0001). The red dots are the reported a-priori expected mortality rates used for sample size calculations