A most stubborn bias: no adjustment method fully resolves confounding by indication in observational studies

Abstract

Objective: To evaluate the effectiveness of methods that control for confounding by indication, we compared breast cancer recurrence rates among women receiving adjuvant chemotherapy with those who did not.

Study design and setting: In a medical record review-based study of breast cancer treatment in older women (n=1798) diagnosed between 1990 and 1994, our crude analysis suggested that adjuvant chemotherapy was positively associated with recurrence (hazard ratio [HR]=2.6; 95% confidence interval [CI]=1.9, 3.5). We expected a protective effect, so postulated that the crude association was confounded by indications for chemotherapy. We attempted to adjust for this confounding by restriction, multivariable regression, propensity scores (PSs), and instrumental variable (IV) methods.

Results: After restricting to women at high risk for recurrence (n=946), chemotherapy was not associated with recurrence (HR=1.1; 95% CI=0.7, 1.6) using multivariable regression. PS adjustment yielded similar results (HR=1.3; 95% CI=0.8, 2.0). The IV-like method yielded a protective estimate (HR=0.9; 95% CI=0.2, 4.3); however, imbalances of measured factors across levels of the IV suggested residual confounding.

Conclusion: Conventional methods do not control for unmeasured factors, which often remain important when addressing confounding by indication. PS and IV analysis methods can be useful under specific situations, but neither method adequately controlled confounding by indication in this study.

Figure 1. Venn diagram of analytic sample sizes for each adjustment method used to control for confounding by indication in a study of older women with breast cancer

* BOW cohort excluding women with biopsy only (n=22), neoadjuvant chemotherapy (N=3); inconsistent chemotherapy dates (n=13), and women who had a recurrence (n=16) or died (n=6) before the start of adjuvant chemotherapy or 183 days after date of diagnosis. † Restricted to women who are classified as high-risk for recurrence by the 1992 St. Gallen Criteria. ‡ Trimmed the sample to exclude women who did not have overlapping propensity scores. § Excludes 253 women because they were the only patient seen by their surgeon or because they were the chronologically first patient for their surgeon, in our dataset.

Figure 2. Propensity score distribution for adjuvant chemotherapy in older women with breast cancer by quintile. The propensity score analytic sample trimmed the “All exposed” and “All unexposed” categories

Figure 3. Estimates and standard errors for the association between adjuvant chemotherapy and rate of breast cancer recurrence in older women