Review of pragmatic trials found that multiple primary outcomes are common but so too are discrepancies between protocols and final reports

Abstract

Objectives: To describe prevalence of multiple primary outcomes, changes in primary outcomes and target sample sizes between protocols and final reports, and how issues of multiplicity are addressed in pragmatic trials.

Study design and setting: Individually randomized trials labeled as pragmatic, published 2014-2019 in MEDLINE and registered with ClinicalTrials.gov.

Results: We identified 262 final reports and located protocols for 159 (61%); primary outcomes were clearly reported in 145 (91%) protocols and 256 (98%) final reports. Thirty (19%) protocols and 38 (15%) final reports had multiple primary outcomes. Primary outcomes were present and identical in 128 (81%) matched protocol-final reports. Among 140 pairs with target sample sizes reported, 28 (20.0%) reduced their target sample size (mean 543 fewer participants per trial) and 16 (11.4%) increased it (mean 192 more participants per trial). Thirteen (29.5%) provided an explanation. Only 2 of 30 (7%) protocols and 4 of 38 (11%) final reports with co-primary outcomes explained how results would be interpreted in light of multiplicity; 21 of 30 (70%) protocols and 20 of 38 (53%) final reports accounted for co-primary outcomes in power calculations.

Conclusion: Co-primary outcomes are common in pragmatic trials; improved transparency around design and analysis decisions involving co-primary outcomes is required.

Keywords: Multiplicity; Outcome selection; Pragmatic trials; Primary outcome; Sample size calculation.

Fig. 1.

Distribution of target sample sizes, their ratio, and their absolute increase or decrease between protocol and final report. (A) Distribution of target sample sizes (logarithmic scale) for the N = 140 trials with sample sizes in both protocol and final report; the median target sample size in the protocols was 395 (ranging from 80 to 20000) and the median target sample size in final reports was 360 (ranging from 80 to 14224). (B) Classification of the ratio of target sample sizes (Final/Protocol) for N = 140 trials with sample size in both documents: no change (ratio = 1) in target sample size occurred in 96 (68.6%). (C) Absolute change in target sample size (logarithmic scale) for the N = 44 trials with discrepant target sample sizes between protocol and final report: the mean reduction in target sample size was 543 participants and the mean increase was 192 participants.