Best Practices for Gauging Evidence of Causality in Air Pollution Epidemiology

Abstract

The contentious political climate surrounding air pollution regulations has brought some researchers and policy-makers to argue that evidence of causality is necessary before implementing more stringent regulations. Recently, investigators in an increasing number of air pollution studies have purported to have used "causal analysis," generating the impression that studies not explicitly labeled as such are merely "associational" and therefore less rigorous. Using 3 prominent air pollution studies as examples, we review good practices for how to critically evaluate the extent to which an air pollution study provides evidence of causality. We argue that evidence of causality should be gauged by a critical evaluation of design decisions such as 1) what actions or exposure levels are being compared, 2) whether an adequate comparison group was constructed, and 3) how closely these design decisions approximate an idealized randomized study. We argue that air pollution studies that are more scientifically rigorous in terms of the decisions made to approximate a randomized experiment are more likely to provide evidence of causality and should be prioritized among the body of evidence for regulatory review accordingly. Our considerations, although presented in the context of air pollution epidemiology, can be broadly applied to other fields of epidemiology.

Keywords: air pollution; causal inference; causality; design decisions; study design.

Figure 1.

Time-series plots of daily numbers of asthma-related emergency department visits to 12 hospitals in 5 counties in Atlanta, Georgia, for the summer Olympic period in 1996 and for the average of the years 1995 and 1997–2004 (comparison group). “A” denotes the daily number of asthma visits for the days before and after the Olympic period (black time series); “B” denotes the daily number of asthma visits during the Olympic period (blue time series). A′ denotes the daily number of asthma visits for the comparison group. Reproduced from Peel et al. (20) with permission (slightly modified).

Figure 2.

Schematic representation of how to construct a comparison group in the Harvard Six Cities Study (21) by exact matching on measured confounders. “A” denotes a group of individuals who live in a low-pollution area (e.g., Portage, Wisconsin (WI)). “B” denotes a group of individuals who live in a high-pollution area (e.g., Steubenville, Ohio (OH)). These 2 groups are heterogeneous; for example, group B has a larger number of smokers than group A. Say that Jane, a participant in the study, lives in Steubenville (exposed to high pollution in group B), does not smoke, and has a low socioeconomic status. A comparison group would be constructed by matching Jane to a group of women with the same confounding characteristics (the same age, smoking status, and socioeconomic status) who live in Portage (exposed to low pollution in group A). The comparison group is represented by A′, which is a subset of A. PM_2.5, particulate matter less than or equal to 2.5 μm in aerodynamic diameter.