Emulating a Novel Clinical Trial Using Existing Observational Data. Predicting Results of the PreVent Study

Abstract

Rationale: "Target trial emulation" has been proposed as an observational method to answer comparative effectiveness questions, but it has rarely been attempted concurrently with a randomized clinical trial (RCT).Objectives: We tested the hypothesis that blinded analysts applying target trial emulation to existing observational data could predict the results of an RCT.Methods: PreVent (Preventing Hypoxemia with Manual Ventilation during Endotracheal Intubation) was a multicenter RCT examining the effects of positive-pressure ventilation during tracheal intubation on oxygen saturation and severe hypoxemia. Analysts unaware of PreVent's results used patient-level data from three previous trials evaluating airway management interventions to emulate PreVent's eligibility criteria, randomization procedure, and statistical analysis. After PreVent's release, results of this blinded observational analysis were compared with those of the RCT. Difference-in-differences estimates for comparison of treatment effects between the observational analysis and the PreVent trial are reported on the absolute scale.Results: Using observational data, we were able to emulate PreVent's randomization procedure to produce balanced groups for comparison. The lowest oxygen saturation during intubation was higher in the positive-pressure ventilation group than the no positive-pressure ventilation group in the observational analysis (n = 360; mean difference = 1.8%; 95% confidence interval [CI] = -1.0 to 4.6) and in the PreVent trial (n = 401; mean difference = 3.9%; 95% CI = 1.4 to 6.4), though the observational analysis could not exclude no difference. Difference-in-differences estimates comparing treatment effects showed reasonable agreement for lowest oxygen saturation between the observational analysis and the PreVent trial (mean difference = -2.1%; 95% CI = -5.9 to 1.7). Positive-pressure ventilation resulted in lower rates of severe hypoxemia in both the observational analysis (risk ratio = 0.60; 95% CI = 0.38 to 0.93) and in the PreVent trial (risk ratio = 0.48; 95% CI = 0.30 to 0.77). The absolute reduction in the incidence of severe hypoxemia with positive-pressure ventilation was similar in the observational analysis (9.4%) and the PreVent trial (12.0%), though the difference between these estimates had wide CIs (mean difference = 2.5%; 95% CI = -8.0 to 13.6%).Conclusions: Applying target trial emulation methods to existing observational data for the evaluation of a novel intervention produced results similar to those of a randomized trial. These findings support the use of target trial emulation for comparative effectiveness research.

Keywords: causal inference; clinical trials; epidemiology; intubation; target trial emulation.

Figure 1.

CONSORT (Consolidated Standards of Reporting Trials) diagram. PreVent = Preventing Cardiovascular Collapse with Administration of Fluid Resuscitation before Tracheal Intubation; Sat = saturation.

Figure 2.

Lowest arterial oxygen saturation and severe hypoxemia in treatment versus control groups. (A) Dot plots showing the distributions of lowest oxygen saturation by group. Bold line represents the median; thinner lines represent the 25th and 75th percentile. (B) Bar charts showing the proportion with severe hypoxemia (lowest oxygen saturation <80%) by group. Error bars represent 95% confidence interval limits. PPV = positive pressure ventilation; PreVent = Preventing Hypoxemia with Manual Ventilation during Endotracheal Intubation.