Elucidating Analytic Bias Due to Informative Cohort Entry in Cancer Clinico-genomic Datasets

Abstract

Background: Oncologists often order genomic testing to inform treatment for worsening cancer. The resulting correlation between genomic testing timing and prognosis, or "informative entry," can bias observational clinico-genomic research. The efficacy of existing approaches to this problem in clinico-genomic cohorts is poorly understood.

Methods: We simulated clinico-genomic cohorts followed from an index date to death. Subgroups in each cohort who underwent genomic testing before death were "observed." We varied data generation parameters under four scenarios: (i) independent testing and survival times; (ii) correlated testing and survival times for all patients; (iii) correlated testing and survival times for a subset of patients; and (iv) testing and mortality exclusively following progression events. We examined the behavior of conditional Kendall tau (Tc) statistics, Cox entry time coefficients, and biases in overall survival (OS) estimation and biomarker inference across scenarios.

Results: Scenario #1 yielded null Tc and Cox entry time coefficients and unbiased OS inference. Scenario #2 yielded positive Tc, negative Cox entry time coefficients, underestimated OS, and biomarker associations biased toward the null. Scenario #3 yielded negative Tc, positive Cox entry time coefficients, and underestimated OS, but biomarker estimates were less biased. Scenario #4 yielded null Tc and Cox entry time coefficients, underestimated OS, and biased biomarker estimates. Transformation and copula modeling did not provide unbiased results.

Conclusions: Approaches to informative clinico-genomic cohort entry, including Tc and Cox entry time statistics, are sensitive to heterogeneity in genotyping and survival time distributions.

Impact: Novel methods are needed for unbiased inference using observational clinico-genomic data.

Figure 1:

Diagram depicting informative left truncation. Figure 1A: Example disease trajectory for a patient from a full “true” cohort in which next-generation sequencing (NGS) is not required for inclusion, and all time after diagnosis is at risk for mortality. Figure 1B: Example disease trajectory for a patient from the “observed” subset in which NGS is required for inclusion and may be performed for worsening clinical status. Here, the time prior to NGS is “immortal,” because patients who died without NGS would not have been observed, but the time after NGS is a period of excess mortality risk compared to the full disease trajectory, because NGS was performed for worsening clinical status.

Figure 2:

Results for scenario #1, corresponding to uniformly distributed genomic testing times without correlation with survival times. Inference results refer to whether a biomarker association in the correct direction was detected at two-sided p < 0.05 in the observed cohort. (a) Observed biomarker inference results as a function of number of patients observed and calculated biomarker effect in a random sample of the full (true) cohort equal in size to the size of the observed cohort; (b) observed biomarker hazard ratio versus biomarker hazard ratio measured in the true cohort; (c) distribution of Kendall’s tau statistics among simulated observed cohorts; (d) distribution of Cox coefficients for genomic testing time across simulated observed cohorts; (e) distribution of observed median survival time minus true median survival time across cohorts (simple risk set adjustment); (f) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment minus log hazard ratio for biomarker effect measured in the true cohort across simulated cohorts; (g) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment and entry time as a covariate minus log hazard ratio for biomarker effect measured in the true cohort across simulations

Figure 3:

Results for scenario #2, corresponding to testing times normally distributed around survival times for all patients. Inference results refer to whether a biomarker association in the correct direction was detected at two-sided p < 0.05 in the observed cohort. (a) observed biomarker inference results as a function of number of patients observed and calculated biomarker effect in a random sample of the full (true) cohort equal in size to the size of the observed cohort; (b) observed biomarker hazard ratio versus biomarker hazard ratio measured in the true cohort; (c) distribution of conditional Kendall’s tau statistics among simulated observed cohorts; (d) distribution of Cox coefficients for entry time among simulated observed cohorts; (e) distribution of observed median survival times minus true survival times among simulated cohorts (simple risk set adjustment); (f) distribution of observed log hazard ratio for biomarker effect from Cox models minus log hazard ratio for biomarker effect measured in the true cohort across simulations (simple risk set adjustment); (g) distribution of observed median survival times minus true median survival times among simulated cohorts (transformation modeling); (h) distribution of observed median survival times minus true median survival times among simulated cohorts (copula modeling); (i) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment and entry time as a covariate minus log hazard ratio for biomarker effect measured in the true cohort across simulations

Figure 4:

Results for scenario #3, corresponding to cohorts in which one subgroup within each cohort undergoes early genomic testing independent of survival time, while a second subgroup exhibits testing normally distributed around survival times. Inference results refer to whether a biomarker association in the correct direction was detected at two-sided p < 0.05 in the observed cohort. (a) Observed biomarker inference results as a function of number of patients observed and calculated biomarker effect in a random sample of the full (true) cohort equal in size to the size of the observed cohort; (b) observed biomarker hazard ratio versus biomarker hazard ratio measured in the true cohort; (c) distribution of conditional Kendall’s tau statistics among simulated observed cohorts as a function of the proportion of patients genotyped “early,” or independently of clinical risk; (d) distribution of Kendall’s tau statistics among simulated observed cohorts; (e) distribution of Cox coefficients for genomic testing time among simulated observed cohorts; (f) distribution of observed median survival times minus true median survival times among simulated cohorts (simple risk set adjustment); (g) distribution of observed log hazard ratio for biomarker effect from Cox models minus log hazard ratio for biomarker effect (simple risk set adjustment) (h) distribution of observed median survival times minus true median survival times among simulated cohorts (transformation modeling); (i) distribution of observed median survival times minus true median survival times among simulated cohorts (copula modeling); (j) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment and entry time as a covariate minus log hazard ratio for biomarker effect measured in the true cohort across simulations

Figure 5:

Results for Scenario #4, corresponding to cohorts in which genomic testing events and survival events are each exponentially distributed following progression events. Inference results refer to whether a biomarker association in the correct direction was detected at two-sided p < 0.05 in the observed cohort. (a) Observed biomarker inference results as a function of number of patients observed and calculated biomarker effect in a random sample of the full (true) cohort equal in size to the size of the observed cohort; (b) observed biomarker hazard ratio versus biomarker hazard ratio measured in the true cohort; (c) distribution of Kendall’s tau statistics among simulated observed cohorts; (d) distribution of Cox coefficients for entry time among simulated observed cohorts; (e) distribution of observed median survival times minus true median survival times among simulated cohorts (simple risk set adjustment); (f) distribution of observed median survival times minus true median survival times among simulated cohorts (transformation modeling); (g) distribution of observed median survival times minus true median survival times among simulated cohorts (copula modeling); (h) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment minus log hazard ratio for biomarker effect measured in the true cohort across simulations; (i) distribution of observed log hazard ratio for biomarker effect from Cox models with risk set adjustment and entry time as a covariate minus log hazard ratio for biomarker effect measured in the true cohort across simulations