Meta-analyses: what they can and cannot do

Abstract

Meta-analyses overcome the limitation of small sample sizes or rare outcomes by pooling results from a number of individual studies to generate a single best estimate. As long as a meta-analysis is not limited by poor quality of included trials, unexplainable heterogeneity and/or reporting bias of individual trials, meta-analyses can be instrumental in reliably demonstrating benefit or harm of an intervention when results of individual randomised controlled trials are conflicting or inconclusive. Therefore meta-analyses should be conducted as part of a systematic review, i.e., a systematic approach to answer a focused clinical question. Important features of a systematic review are a comprehensive, reproducible search for primary studies, selection of studies using clear and transparent eligibility criteria, standardised critical appraisal of studies for quality, and investigation of heterogeneity among included studies. Cumulative meta-analysis may prevent delays in the introduction of effective treatments and may allow for early detection of harmful effects of interventions. As opposed to meta-analysis based on aggregate study data, individual patient data meta-analyses offer the advantage to use standardised criteria across trials and reliably investigate subgroup effects of interventions. Network meta-analysis allows the integration of data from direct and indirect comparisons in order to compare multiple treatments in a comprehensive analysis and determine the best treatment among several options. We conclude that meta-analysis has become a popular, versatile, and powerful tool. If rigorously conducted as part of a systematic review, it is essential for evidence-based decision making in clinical practice as well as on the health policy level.