Integrating heterogeneous pieces of evidence in systematic reviews

Abstract

Researchers preparing systematic reviews often encounter various types of evidence, which can generally be categorized as direct or indirect. The former directly relates an exposure, diagnostic strategy, or therapeutic intervention to the occurrence of a principal health outcome. Evidence is indirect if two or more bodies of evidence are required to relate the exposure, diagnostic strategy, or intervention to the principal health outcome. Heterogeneity of data sources complicates integration of both direct and indirect evidence. Participants in different studies may have a wide spectrum of baseline risk and sociodemographic and cultural characteristics. A variety of formulations and intensities of exposures, diagnostic strategies, and interventions, as well as diversity in the selection and definition of control groups, may be encountered. Outcome measures may be different, and similar outcomes may be measured or reported differently. Heterogeneity of study designs and of methodologic features and quality within a given design may be found. The effective integration of direct and indirect evidence requires development of explicit models that serve as analytic frameworks for linking the important pieces of evidence. A model can be viewed as a series of subquestions, with each important subquestion warranting a systematic review. Several subjective and quantitative methods can then be used to integrate the evidence. Tabular displays of major findings and strength of evidence for each subquestion can help reviewers, patients, and providers to integrate the differing research findings and draw reasonable conclusions. Various quantitative techniques, such as decision analysis and the confidence profile method, are also available. No single integration approach is clearly superior, none obviates uncertainty, and all underscore the role of careful judgment in integrating evidence.