Meta-analysis of small randomized controlled trials in surgery may be unreliable

Abstract

Background: Meta-analysis of randomized controlled trials (RCTs) should provide reliable evidence about the effects of interventions. This may be less reliable when only small trials are available.

Methods: The sample size was determined for all surgical RCTs included in Cochrane Collaboration systematic reviews. The difficulty in interpreting meta-analysis of small trials is illustrated using two specific reviews.

Results: The typical sample size for surgical RCTs was small with a median of only 87 participants. Only 39.8 per cent had adequate prerandomization treatment allocation concealment. In both systematic reviews that were assessed in detail, statistically significant early results from meta-analysis of several small RCTs did not reliably predict the results of subsequent RCTs.

Conclusion: Surgical RCTs tend to be small and underpowered. Meta-analysis of such trials does not necessarily produce reliable results.