Rethinking Meta-Analysis: Applications for Air Pollution Data and Beyond

Abstract

Meta-analyses offer a rigorous and transparent systematic framework for synthesizing data that can be used for a wide range of research areas, study designs, and data types. Both the outcome of meta-analyses and the meta-analysis process itself can yield useful insights for answering scientific questions and making policy decisions. Development of the National Ambient Air Quality Standards illustrates many potential applications of meta-analysis. These applications demonstrate the strengths and limitations of meta-analysis, issues that arise in various data realms, how meta-analysis design choices can influence interpretation of results, and how meta-analysis can be used to address bias and heterogeneity. Reviewing available data from a meta-analysis perspective can provide a useful framework and impetus for identifying and refining strategies for future research. Moreover, increased pervasiveness of a meta-analysis mindset-focusing on how the pieces of the research puzzle fit together-would benefit scientific research and data syntheses regardless of whether or not a quantitative meta-analysis is undertaken. While an individual meta-analysis can only synthesize studies addressing the same research question, the results of separate meta-analyses can be combined to address a question encompassing multiple data types. This observation applies to any scientific or policy area where information from a variety of disciplines must be considered to address a broader research question.

Keywords: Air pollutants; bias; data synthesis; heterogeneity; meta-analysis.

Figure 1

Steps of a meta-analysis. Adapted from CRD.

Figure 2

Forest plot showing the difference in responses to airway challenge provocative doses following exposure of asthmatics to NO₂ vs. air. This figure illustrates the types of meta-analysis findings that can be graphically illustrated in a forest plot, e.g., the average change per dose from each study (the central dots within the squares), the proportional weights used in each meta-analysis (the squares), and summary measures and confidence intervals for each dose level (horizontal lines) and the overall study (the center lines and lateral tips of the diamonds). The results from individual studies and study combinations can be compared with the vertical lines (with the solid line indicating no effect and the dotted line indicating the overall summary measure) to assess whether results are consistent across studies or whether a dose-response relationship appears to exist. Adapted from Goodman et al.

Figure 3

Association between NO₂ exposure and airway hyper-responsiveness in asthmatics based on meta-regression of the difference between airway challenge provocative dose following exposure to NO₂ vs. air. This figure illustrates the use of a bubble plot to display meta-regression results. Each circle represents the findings from one study at a given exposure, while the area of each circle is proportional to the weight given to each measure in the meta-regression. Adapted from Goodman et al.

Figure 4

Percent increase in total mortality associated with a 10 μg/m³ increase in PM₁₀ in 90 NMMAPS cities, with 95% confidence intervals and grouped by region. This figure illustrates another approach to displaying results using a forest plot. The open circles represent specific cities. Summary estimates based on two methodologies are shown, in bold, to the right of the individual city results for each region (as delineated by the dotted lines) and for national estimates (shown on the far right). Adapted from Samet et al.

Figure 5

Distributions of summary log-relative risks of all-cause mortality associated with a 10-ppb increase in O₃ in 95 cities (NMMAPS) compared to a meta-analysis of 11 U.S. estimates. The multicity results yielded a lower and more precise estimate of the overall percent decrease in mortality associated with O₃ exposures than did the meta-analysis based on studies reporting results from single cities. Source: Bell et al.