Implications of Heterogeneity of Treatment Effect for Reporting and Analysis of Randomized Trials in Critical Care

Abstract

Randomized clinical trials (RCTs) are conducted to guide clinicians' selection of therapies for individual patients. Currently, RCTs in critical care often report an overall mean effect and selected individual subgroups. Yet work in other fields suggests that such reporting practices can be improved. Specifically, this Critical Care Perspective reviews recent work on so-called "heterogeneity of treatment effect" (HTE) by baseline risk and extends that work to examine its applicability to trials of acute respiratory failure and severe sepsis. Because patients in RCTs in critical care medicine-and patients in intensive care units-have wide variability in their risk of death, these patients will have wide variability in the absolute benefit that they can derive from a given therapy. If the side effects of the therapy are not perfectly collinear with the treatment benefits, this will result in HTE, where different patients experience quite different expected benefits of a therapy. We use simulations of RCTs to demonstrate that such HTE could result in apparent paradoxes, including: (1) positive trials of therapies that are beneficial overall but consistently harm or have little benefit to low-risk patients who met enrollment criteria, and (2) overall negative trials of therapies that still consistently benefit high-risk patients. We further show that these results persist even in the presence of causes of death unmodified by the treatment under study. These results have implications for reporting and analyzing RCT data, both to better understand how our therapies work and to improve the bedside applicability of RCTs. We suggest a plan for measurement in future RCTs in the critically ill.

Keywords: acute respiratory failure; heterogeneity of treatment effect; precision medicine; randomized clinical trials; sepsis.

Figure 1.

Scenario 1: heterogeneity of treatment effect in positive trials. This figure shows the results of the 8,706 simulated trials from scenario 1 (see main text) that showed a statistically significant benefit of the therapy in patients with acute respiratory failure requiring mechanical ventilation. Nonetheless, the lowest decile of risk showed very little effect. The upper panel shows the results in each arm, stratified by decile of risk; the lower panel shows the net effect: the rate of mortality in the control arm minus the rate in the treated arm.

Figure 2.

Scenario 2: heterogeneity of treatment effect in negative trials. This figure shows the results of the 4,504 simulated trials from scenario 2 (see main text) that failed to show a statistically significant benefit of the therapy in patients with acute respiratory failure requiring mechanical ventilation. Nonetheless, the highest deciles of risk showed consistent benefit (lower mortality in treated arm than in the control arm). The upper panel shows the results in each arm, stratified by decile of risk; the lower panel shows the net effect: the rate of mortality in the control arm minus the rate in the treated arm.