Deriving treatment recommendations from evidence within randomized trials. The role and limitation of meta-analysis

Abstract

Meta-analysis is commonly used in reviews of the effectiveness of medical technologies, but this approach has not been used in direct support of guidelines development groups. This paper describes the approach of the North of England Guidelines Development Project in describing the evidence using meta-analyses that were conducted explicitly to address questions on the choice of therapy raised by the guidelines development groups. Particular emphasis is placed on the context within which the contributing trials were conducted and the extent to which systematic differences between trials (heterogeneity) was observed, described, and explained. There is a trade-off between internal and external validity for different metrics when presenting the results of trials. More interpretable metrics, such as risk differences or weighted mean differences, are confounded by study design issues and strong assumptions. More robust measures such as odds ratios or standardized weighted mean differences are difficult to interpret physically. Individual patient data may prove particularly helpful in addressing pivotal questions on the magnitude of effects of interventions, though accessing and reanalyzing these data requires a substantial investment in time and other resources.