Bias magnification in ecologic studies: a methodological investigation

Abstract

Background: As ecologic studies are often inexpensive to conduct, consideration of the magnitude and direction of ecologic biases may be useful in both study design and sensitivity analysis of results. This paper examines three types of ecologic bias: confounding by group, effect measure modification by group, and non-differential exposure misclassification.

Methods: Bias of the risk difference on the individual and ecologic levels are compared using two-by-two tables, simple equations, and risk diagrams. Risk diagrams provide a convenient way to simultaneously display information from both levels.

Results: Confounding by group and effect measure modification by group act in the same direction on the individual and group levels, but have larger impact on the latter. The reduction in exposure variance caused by aggregation magnifies the individual level bias due to ignoring groups. For some studies, the magnification factor can be calculated from the ecologic data alone. Small magnification factors indicate little bias beyond that occurring at the individual level. Aggregation is also responsible for the different impacts of non-differential exposure misclassification on individual and ecologic studies.

Conclusion: The analytical tools developed here are useful in analyzing ecologic bias. The concept of bias magnification may be helpful in designing ecologic studies and performing sensitivity analysis of their results.

Figure 1

Risk diagram illustrating Table 1. We summarize individual-level information for a group with a solid black line, ecologic data with a solid black dot. The line connects the risk in the unexposed (q = 0.2 at x = 0) with the risk in the exposed (0.4 at x = 1) and has slope equal to the risk difference b. The ecologic data are the average exposure X and average risk Y for the group.

Figure 2

Nonlinearity causes error during aggregation. The ecologic data point (X, Y) will generally not fall on the risk function when the latter is non-linear as shown here. The amount of error depends on the curvature of the risk function and the exposure distribution, but is bounded above by the line connecting the risks at the minimum and maximum exposures. This error can lead to pure specification bias.

Figure 3

Loss of information is a fundamental problem of ecologic studies. Many sets of individual-level information (lines, interiors of two-by-two tables) generate the same ecologic data (dot, table margins). Only some possible lines are shown.

Figure 4

Confounding by group, illustrating Table 2. A) Individual level: The solid black lines describing the individual-level information in the two groups are parallel (same risk differences b) but have different intercepts (different background risks q₀ ≠ q₁). The crude estimate of the risk difference b_c is confounded (blue line). B) Group level: The ecologic estimate of the risk difference b_e is the slope of the red line through the two ecologic data points. Massive confounding has occurred, but we can't tell this from the ecologic data alone. C) Comparison of results on the two levels: The ecologic estimate of the risk difference b_e is much more biased than the crude individual-level estimate b_c. Both biases are in the same direction.

Figure 5

Confounding by group on the individual and group level. A, B, C) Suppose average exposures are the same, but the difference between the background risks (q_i) decreases. Confounding by group decreases on both the individual and group levels with constant proportionality factor M. D, E, F) Suppose background risks (q_i) are the same, but the difference between the average exposures decreases. Confounding by group decreases on the individual level, but increases on the ecologic level because of the large increase in M.

Figure 6

Effect modification of the risk difference by group, illustrating Table 3. The solid black lines describing the individual-level information for the two groups have the same intercept (background risk q) but different slopes (risk differences b₀ ≠ b₁). The crude estimate of the risk difference b_c (blue line) lies between these two extremes. Relative to b_w, the ecologic estimate of the risk difference b_e (red line) is far more biased than the crude individual-level estimate b_c. Both biases are in the same direction. b_w (purple line) is the weighted average of the risk differences used in the bias magnification equation.

Figure 7

Effect on M of different within-group exposure distributions. The magnification factor M decreases when the within-group exposure variance is reduced, keeping the between-group variance constant (the X_i do not change between rows).

Figure 8

Non-differential exposure misclassification (NDEM), illustrating Table 5. A. If there are no other sources of bias, the ecologic- and individual-level analyses of the correct data are the same. B. Suppose the dichotomous exposure data are misclassified with the same sensitivity and specificity in each group. Then the individual-level result (blue) is biased toward the null and the ecologic result (red) is biased away from the null. The average risks (Y_i) in each group are unchanged but the average exposures move closer together. This causes the resulting ecologic regression line to have higher slope.

Figure 9

Bias magnification and inflation. The ecologic bias (b_e - b_w) equals the individual-level bias (b_c - b_w) – due to confounding by group and effect modification of the risk difference by group – multiplied by the magnification factor M, assuming no other sources of ecologic bias. The ecologic bias also equals the sum of the individual-level bias due to ignoring groups and the bias caused by aggregation. The latter is measured by F, the inflation factor, equal to M - 1.