A review of cluster randomized trials found statistical evidence of selection bias

Abstract

Objectives: To assess markers of selection bias risk in a sample of recently published cluster randomized controlled trials compared with individually randomized trials.

Study design and setting: We used OVID Medline and the online archives of the Journal of the American Medical Association to search for cluster randomized trials published between January 2015 and June 2017 from four high-impact journals and compared them to a matched sample of individually randomized trials.

Results: We identified 23 cluster trials: 57% (n = 13) described a robust allocation method and 17% (n = 4) recruited all participants before randomization. Four (17%), eight (35%), and 11 (48%) were classified as at low, medium, and high bias risk, respectively. Meta-analysis showed significant age imbalance (-0.05, 95% CI = -0.057 to -0.043, I² = 93.2%) in cluster trials, while the matched individually randomized trials showed no imbalance (0.005, 95% CI = -0.026 to 0.035, I² = 0%). Cluster trials finding a statistically significant outcome in their primary measure showed a larger age imbalance (0.082, 95% CI = -0.091 to -0.073, I² = 87%) than trials finding a nonstatistically significant outcome (0.022, 95% CI = 0.008 to 0.035, I² = 83%).

Conclusions: There is strong evidence in this sample of an effect of selection bias seen in an imbalance in baseline participant age, something not seen in a comparable sample of individually randomized trials.

Keywords: Baseline characteristics; Cluster randomized controlled trials; Cluster trials; Meta-analysis; Randomized controlled trials; Selection bias.