Type-II error rates (beta errors) of randomized trials in orthopaedic trauma

Abstract

Background: Although an investigator may limit bias through randomization, concealment of patient allocation, and blinding, the results of randomized trials may be less convincing when the sample size is not sufficiently large to reveal a true difference between treatment groups. When the sample size is small, randomized trials are subject to beta errors (type-II errors)--that is, the probability of concluding that no difference between treatment groups exists when, in fact, there is a difference. The purpose of this study of randomized trials involving fracture care published between 1968 and 1999 was twofold: (1) to evaluate type-II error rates and study power (1 - beta) for the primary outcomes and (2) to identify whether investigators clearly identified the primary and secondary outcomes.

Methods: To be eligible, studies were required to (1) be published in English, (2) be described as a randomized trial, (3) involve the care of adult patients with fractures, treated either operatively or nonoperatively, and (4) contain sufficient outcome information to enable study power to be calculated. Computer database searches were performed independently by two investigators to identify all potentially relevant study titles. Additional strategies to identify articles included (1) hand searches of selected orthopaedic journals from 1989 to 1999, (2) searches of the bibliographies of potentially relevant articles, and (3) review by content experts to identify missing studies. For each study, a standard power calculation was performed on the primary and secondary outcomes. For those studies in which the primary outcome was not explicitly reported, the most clinically relevant measure was chosen by consensus. Acceptable study power was agreed a priori to be > or = 80% (type-I error of < or = 0.20).

Results: We identified 620 potentially relevant citations from MEDLINE, of which only 187 were potentially eligible. We identified nine more articles with other searches, and application of the eligibility criteria to the 196 articles eliminated seventy-nine. Thus, we analyzed 117 studies in which a total of 19,942 patients with orthopaedic trauma had been randomized. Sample sizes ranged from ten to 662 patients (mean and standard deviation, 95 79 patients). The majority (34%) of trials involved the treatment of hip fractures. The mean overall study power among the 117 trials was 24.65% (range, 2% to 99%). The type-II error rate for primary outcomes was 90.52%.

Conclusions: Mean type-II error rates in the orthopaedic trauma trials that we analyzed exceeded accepted standards. Investigators can reduce type-II error rates by performing power and sample-size calculations prior to conducting a trial.