Limits of observational data in determining outcomes from cancer therapy

Abstract

Background: Observational data are used increasingly to assess the effectiveness of therapies. However, selection biases are likely to have an impact on results and threaten the validity of these studies.

Methods: The primary objective of the current study was to explore the effect of selection biases in observational studies of treatment effectiveness in cancer care. Patients were identified from the Surveillance, Epidemiology, and End Results-Medicare linked database. The following groups of patients were included: 5245 men treated with and without androgen deprivation for locally advanced prostate cancer, 43,847 men with active treatment versus observation for low- and intermediate-risk prostate cancer, and 4860 patients with lymph node-positive colon cancer who were treated with and without fluorouracil chemotherapy. Patients were compared by therapy for the outcomes of cancer-specific mortality, other-cause mortality, and overall mortality.

Results: In all comparisons, the observational data produced improbable results. For example, when evaluating outcomes of men who were treated with and without androgen deprivation for locally advanced prostate cancer, men who underwent androgen deprivation had higher prostate cancer mortality (hazard ratio, 1.5; 95% confidence interval, 1.29-1.92) despite clinical trial evidence that this treatment improves cancer mortality. Controlling for comorbidity, extent of disease, and other characteristics by multivariate analyses or by propensity analyses had remarkably small impact on these improbable results.

Conclusions: The current results suggested that the results from observational studies of treatment outcomes should be viewed with caution.

Figure 1

Adjusted overall survival curves from Cox models for men with localized prostate cancer stratified by therapy and for a matched noncancer control population.

Figure 2

Plot of hazard ratio for death comparing active therapy versus observation in men with localized prostate cancer. CVD indicates cardiovascular disease; COPD, chronic obstructive pulmonary disease; DM, diabetes mellitus.