From Protocols to Publications: A Study in Selective Reporting of Outcomes in Randomized Trials in Oncology

Abstract

Purpose: The decision by journals to append protocols to published reports of randomized trials was a landmark event in clinical trial reporting. However, limited information is available on how this initiative effected transparency and selective reporting of clinical trial data.

Methods: We analyzed 74 oncology-based randomized trials published in Journal of Clinical Oncology, the New England Journal of Medicine, and The Lancet in 2012. To ascertain integrity of reporting, we compared published reports with their respective appended protocols with regard to primary end points, nonprimary end points, unplanned end points, and unplanned analyses.

Results: A total of 86 primary end points were reported in 74 randomized trials; nine trials had greater than one primary end point. Nine trials (12.2%) had some discrepancy between their planned and published primary end points. A total of 579 nonprimary end points (median, seven per trial) were planned, of which 373 (64.4%; median, five per trial) were reported. A significant positive correlation was found between the number of planned and nonreported nonprimary end points (Spearman r = 0.66; P < .001). Twenty-eight studies (37.8%) reported a total of 65 unplanned end points; 52 (80.0%) of which were not identified as unplanned. Thirty-one (41.9%) and 19 (25.7%) of 74 trials reported a total of 52 unplanned analyses involving primary end points and 33 unplanned analyses involving nonprimary end points, respectively. Studies reported positive unplanned end points and unplanned analyses more frequently than negative outcomes in abstracts (unplanned end points odds ratio, 6.8; P = .002; unplanned analyses odd ratio, 8.4; P = .007).

Conclusion: Despite public and reviewer access to protocols, selective outcome reporting persists and is a major concern in the reporting of randomized clinical trials. To foster credible evidence-based medicine, additional initiatives are needed to minimize selective reporting.

Fig 1.

Search strategy for and baseline characteristics of randomized trials included in the study. A search for randomized trials returned 252 published trials, of which 74 were included in the current analysis. Reports excluded for miscellaneous reasons were those that were reviews, were published outside the specified time period, or were not oncology related.

Fig 2.

Reporting of planned nonprimary end points (NPEPs), unplanned end points (UEPs), and unplanned analyses (UPAs) in randomized trials. (A) Positive correlation between the number of planned NPEPs and unreported NPEPs. (B) Studies with greater than six NPEPs had more unreported NPEPs than studies with six or fewer NPEPs. (C) Conversely, studies with unreported NPEPs had larger numbers of planned NPEPs than studies that reported all of their NPEPs. (D and E) A higher number of positive UEPs and UPAs than negative UEPs and UPAs were reported in abstracts.

Fig 3.

Recommendations for Comprehensive Outcomes Reporting in Randomized Evidence (CORRE). CORRE table (as a supplementary table) should include a list of all outcomes analyzed along with their reporting statuses, reasons for not reporting outcomes, and exploration of unplanned outcomes. AE, adverse events; ANOVA, analysis of variance; CMH, Cochran-Mantel-Haenszel test; CR, complete response; CRP, C-reactive protein; CT, chemotherapy; CTCAE, Common Terminology Criteria for Adverse Events; ECOG PS, Eastern Cooperative Oncology Group performance status; FACT-G, Functional Assessment of Cancer Therapy–General Scale; ITT, intention-to-treat; KM, Kaplan-Meier; LDH, lactate dehydrogenase; NA, not applicable; NLR, neutrophil-lymphocyte ratio; ORR, objective response rate; OS, overall survival; PD, progressive disease; PFS, progression-free survival; PP, per-protocol; PR, partial response; QOL, quality of life; SD, stable disease; TTP, time to progression.