Preventable sources of bias in subgroup analyses and secondary outcomes of randomized trials

Abstract

Background: Randomized controlled trials are the gold standard for determining treatment efficacy in medicine. To deter harmful practices such as p-hacking and hypothesizing after the results are known, any analysis of subgroups and secondary outcomes must be documented and pre-specified. However, they can still introduce bias (and routinely do) if they are not treated with the same consideration as the primary analysis.

Methods: We describe several sources of bias that affect subgroup and secondary outcome analyses using published randomized trials and causal directed acyclic graphs (DAGs).

Results: We use the RECOVERY and START trials to elucidate sources of bias in analyses of subgroups and secondary outcomes. Chance imbalance can occur if the distribution of prognostic variables is not sought for any given subgroup analysis as for the main analysis. This differential distribution of prognostic variables can also occur in analyses of secondary outcomes. Selection bias can occur if the subgroup variable is causally related to staying in the trial. Given loss to follow up is not normally addressed in subgroups, attrition bias can pass unnoticed in these cases. In every case, the solution is to take the same considerations for these analyses as we do for primary analyses.

Conclusions: Approval of treatments and clinical decisions can occur based on results from subgroup or secondary outcome analyses. Thus, it is important to give them the same treatment as primary analyses to avoid preventable biases.

Keywords: Confounding; Randomized trials; Secondary analyses; Selection bias.