Strengths and limitations of meta-analysis: larger studies may be more reliable

Abstract

Meta-analysis of randomized controlled trials combines information from independent studies that address a similar question to provide more reliable estimates of treatment effects. At the present time, the methodology and usefulness of meta-analysis is under scrutiny. In the first part of this paper, we summarize the limitations of meta-analysis and make suggestions for improvements. In the second part, we illustrate strengths and limitations using examples of meta-analyses and subsequent large trials that address the same question. We develop the hypothesis that the size of the meta-analysis may be a useful measure of reliability. Small meta-analyses (i.e., those with less than 200 outcome events) may only be useful for summarizing the available information and generating hypotheses for future research. The results of small meta-analyses should be regarded with caution, even if the p value shows extreme statistical significance. Larger meta-analyses (i.e., those with several hundred events) are likely to be more reliable and may be clinically useful. Well-conducted meta-analyses of large trials using individual patient data may provide the best estimates of treatment effects in the cohort overall and in clinically important subgroups.