A simulation study evaluating approaches to the analysis of ordinal outcome data in randomized controlled trials in traumatic brain injury: results from the IMPACT Project

Abstract

Background: Clinical trials in traumatic brain injury have a disappointing track record, with a long history of 'negative' Phase III trials. One contributor to this lack of success is almost certainly the low efficiency of the conventional approach to the analysis, which discards information by dichotomizing an ordinal outcome scale.

Purpose: Our goal was to evaluate the potential efficiency gains, which can be achieved by using techniques, which extract additional information from ordinal outcome data - the proportional odds model and the sliding dichotomy. In addition, we evaluated the additional efficiency gains, which can be achieved through covariate adjustment.

Methods: The study was based on simulations, which were built around a database of patient-level data extracted from eight Phase III trials and three observational studies in traumatic brain injury. Two different putative treatment effects were explored, one which followed the proportional odds model, and the other which assumed that the effect of the intervention was to reduce the risk of death without changing the distribution of outcomes within survivors. The results are expressed as efficiency gains, reported as the percentage reduction in sample size that can be used with the ordinal analyses without loss of statistical power relative to the conventional binary analysis.

Results: The simulation results show substantial efficiency gains. Use of the sliding dichotomy allows sample sizes to be reduced by up to 40% without loss of statistical power. The proportional odds model gives modest additional gains over and above the gains achieved by use of the sliding dichotomy.

Limitations: As with any simulation study, it is difficult to know how far the findings may be extrapolated beyond the actual situations that were modeled.

Conclusions: Both ordinal techniques offer substantial efficiency gains relative to the conventional binary analysis. The choice between the two techniques involves subtle value judgments. In the situations examined, the proportional odds model gave efficiency gains over and above the sliding dichotomy, but arguably, the sliding dichotomy is more intuitive and clinically appealing.