Meta-analysis: Methods, strengths, weaknesses, and political uses

Abstract

The general methodology, strengths and weaknesses, and political uses of meta-analysis are examined. As a systematic study of all studies that have been conducted to answer a specific question or hypothesis, meta-analysis is strong in revealing structural flaws and sources of bias in primary research and in posing promising research questions for future study. It cannot exceed, however, the limits of what is reported by primary researchers. Meta-analysis is particularly challenged to quantify the size of a common effect of treatment across reported trials because of (1) the clinical diversity of the trials and (2) the myriad of potential differences among patients with varying characteristics within the trials. Without access to the original data of reported trials, meta-analysis cannot overcome the bias of underpowered trials toward overstatement of the size of main treatment effects, nor the tendency for such trials to falsely conclude there were no statistically significant adverse events. Although severely compromised by ghost-written or honorary-authored reports of primary research, meta-analysis can make use of its methods to focus on the conflicts of interest and likely sources of bias of such research and make known what precautions should be taken by would-be consumers. Examples show how meta-analysis has clarified thinking about the off-label use of selective serotonin reuptake inhibitors for treating child and adolescent depression, use of low-tidal volume respirator assistance for acute lung injury and acute respiratory distress syndrome patients, and the long-term use of COX-2 inhibitors for relieving arthritic pain. Recommendations are made for Congressional action.