Impervious to Randomness: Confounding and Selection Biases in Randomized Clinical Trials

Abstract

The random allocation of therapies in randomized clinical trials is a powerful tool that removes all confounding biases that can affect treatment assignment. However, confounders influencing mediators of the treatment effect are unaffected by randomization and should be considered during trial design and statistical modeling.Examples of such mediators include biomarkers predictive of response to targeted therapies in oncology. Patient selection for such biomarkers is prudent in clinical trials. Conversely, prognostic information on outcome heterogeneity can be derived from observational datasets that include more representative populations. The fusion of experimental and observational data can then allow patient-specific inferences.

Keywords: Confounding; directed acyclic graphs; mediation analysis; representativeness; selection bias.

Figure 1.

Example causal relationships represented by directed acyclic graphs. (A) Confounders (highlighted in red) are variables that influence both the exposure and the outcome. These variables confound the exposure-outcome relationship and need to be adjusted for when estimating the effect of the exposure on the outcome. (B) Prognostic variables (highlighted in blue) influence only the outcome. Adjusting for such variables improves the power of null hypothesis tests used to evaluate the effect of the exposure on the outcome. (C) Colliders are variables that are influenced by both the exposure and the outcome. Adjusting for colliders introduces a bias called “collider bias” that distorts the association between the exposure and the outcome. (D) Mediators transmit the effect of the exposure on the outcome. (E) Variables that influence both mediators and the outcome are confounders of the mediator-outcome relationship. (F) Instrumental variables can be used to estimate the effect of the exposure on the outcome in cases where the confounders of the exposure-outcome relationship are unknown or unmeasurable. The instrumental variable should i) be independent of the confounder, ii) influence the outcome only via the exposure, iii) be strongly associated with the exposure, ideally with a correlation coefficient of one. (G) Selection nodes point to variables that differ across populations. In this example, the selection node indicates that the prognostic variable shifts across populations. Thus, to estimate the effect of a specific value of the prognostic variable on the outcome we need datasets from populations that include that value. For example, to estimate the prognostic effect of being aged ≥ 65 years old, we need to study populations of that age group. Selection biases that exclude this age group hinder the estimation of the outcome for these patients.

Figure 2.

Directed acyclic graphs for a hypothetical trial of systemic therapy for metastatic non-small cell lung cancer. Oncogenic EGFR signaling is the mediator for the effect of treatment assignment (exposure) on the outcome of overall survival. Red boxes highlight confounders, whereas blue boxes highlight prognostic variables that can improve the power of null hypothesis tests used to evaluate the effect of treatment assignment on overall survival. (A) In the non-randomized version of the trial, confounders bias the effect of treatment assignment on overall survival. (B) The effect of confounders on treatment assignment is removed by randomizing the treatment assignment, as noted by the red cross-mark. These former confounding variables can no longer influence treatment assignment and now act only as prognostic variables that influence overall survival. (C) As noted by the orange cross-marks, random sampling can remove the selection biases induced by the criteria used for trial enrollment. However, removing these selection biases does not affect the confounding induced by EGFR mutation status on the relationship between oncogenic EGFR signaling and overall survival. (D) Conversely, as noted by the purple cross-marks, selecting only for patients with oncogenic EGFR mutation removes its confounding on oncogenic EGFR signaling and overall survival. The selection nodes S1, S2, and S3 point to variables that may differ between the clinical trial population and the population of patients physicians encounter in clinic. The effect on overall survival of the prognostic variables denoted by S1 and S2 can be directly estimated from observational datasets.