Treatment success in cancer: industry compared to publicly sponsored randomized controlled trials

Abstract

Objective: To assess if commercially sponsored trials are associated with higher success rates than publicly-sponsored trials.

Study design and settings: We undertook a systematic review of all consecutive, published and unpublished phase III cancer randomized controlled trials (RCTs) conducted by GlaxoSmithKline (GSK) and the NCIC Clinical Trials Group (CTG). We included all phase III cancer RCTs assessing treatment superiority from 1980 to 2010. Three metrics were assessed to determine treatment successes: (1) the proportion of statistically significant trials favouring the experimental treatment, (2) the proportion of the trials in which new treatments were considered superior according to the investigators, and (3) quantitative synthesis of data for primary outcomes as defined in each trial.

Results: GSK conducted 40 cancer RCTs accruing 19,889 patients and CTG conducted 77 trials enrolling 33,260 patients. 42% (99%CI 24 to 60) of the results were statistically significant favouring experimental treatments in GSK compared to 25% (99%CI 13 to 37) in the CTG cohort (RR = 1.68; p = 0.04). Investigators concluded that new treatments were superior to standard treatments in 80% of GSK compared to 44% of CTG trials (RR = 1.81; p<0.001). Meta-analysis of the primary outcome indicated larger effects in GSK trials (odds ratio = 0.61 [99%CI 0.47-0.78] compared to 0.86 [0.74-1.00]; p = 0.003). However, testing for the effect of treatment over time indicated that treatment success has become comparable in the last decade.

Conclusions: While overall industry sponsorship is associated with higher success rates than publicly-sponsored trials, the difference seems to have disappeared over time.

Figure 1. PRISMA flow diagram depicting process of identification and selection of studies.

Figure 2. Success rate of GlaxoSmithKline (GSK) compared with National Cancer Institute Canada Clinical Trials Group (CTG) cohort of studies.

(A) Distribution of success rate according to statistical significance of the results for the primary outcome; (B) Distribution of success rate according to investigators' judgments. *Data for one comparison in the GSK cohort were not available to make a decision on investigators' judgments. For ten comparison in the GSK cohort and two comparisons in the CTG cohort data were not available to make a judgment on whether investigators considered the experimental treatment to be a breakthrough ( = fit for adoption as standard of care”). **The results were available in the summary format (unpublished). Therefore, investigator judgments were not possible to assess for 10 comparisons. (C) Forest plot showing quantitative pooling of data on primary outcome for studies conducted by CTG and GSK. The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs).

Figure 3. Distribution of success rate according to the results being statistically significant versus non-significant for the a priori specified primary outcome.

Figure 4. Forest plot of distribution of success rate for outcomes of overall survival, event-free survival, response rate, and treatment relate mortality for GlaxoSmithKline (GSK) and National Cancer Institute Canada Clinical Trials Group (CTG) cohorts.

The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs). Note that unlike for the pooled analysis for all primary outcomes (Fig 2) test of interaction detected no statistically significant difference between subgroups, but the number of comparisons was too small to detect a difference between two cohorts.

Figure 5. Forest plot of pooled data on primary outcome for studies of antiemetic therapies conducted by GlaxoSmithKline (GSK) and National Cancer Institute Canada Clinical Trials Group (CTG).

Given the prevalence of studies involving antiemetic treatments, we include this subgroup analysis. The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs). The test of interaction is statistically non-significant between the two cohorts but the number of trials was small.

Figure 6. Forest plot of sensitivity analysis for distribution of success rate according to methodological quality of included studies for GlaxoSmithKline (GSK) and National Cancer Institute Canada Clinical Trials Group (CTG) cohorts.

The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs).

Figure 7. Forest plot of sensitivity analysis for distribution of success rate according to elements of random error that can potentially impact outcomes for GlaxoSmithKline (GSK) and National Cancer Institute Canada Clinical Trials Group (CTG) cohorts.

The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs). * Represents a statistically significant test for interaction between subgroups. The test for interaction was statistically significant in cohort of GSK trials reporting versus not reporting expected difference for primary outcome.

Figure 8. Forest plot of sensitivity analysis for distribution of success rate according to cancer and treatment type, study design, and choice of control for GlaxoSmithKline (GSK) and National Cancer Institute Canada Clinical Trials Group (CTG) cohorts.

The summary pooled estimate (odds/hazard ratio) is indicated by rectangles, with the lines representing 99% confidence intervals (CIs). * Represents a statistically significant test for interaction between subgroups. Note that the experimental treatments were statistically superior in GSK trials where comparators were placebo or no therapy.

Figure 9. Comparison according to placebo versus active comparator.

(A) Success rate according to statistically significant findings, investigator judgment and meta-analysis in placebo controlled trials. (B) Success rate according to statistically significant findings, investigator judgment and meta-analysis in trials having active treatment as control. Statistically significant results were observed in GlaxoSmithKline (GSK) trials according to all 3 metrics of summarizing treatment success (a proportion of statistically significant results favouring experimental treatment, according to the investigators' judgments and quantitative pooling of data). (B). When the active comparator was used, this was only true for the metrics according to investigators' judgments. Test of interactions in placebo-controlled trials was highly significant in favour of GSK compared to National Cancer Institute Canada Clinical Trials Group (CTG) comparisons (p = 0.001) while it was not significant (p = 0.154) between trials when active control was used as a comparator.

Figure 10. Assessment of the pattern of treatment successes over time.

A) Time series analysis of treatment effect (natural logarithm of hazard ratio [ln HR]). Data are consistent with “white noise” pattern indicates no significant autocorrelation between studies carried out at various time intervals. An ln HR less than 0 indicates superiority of new treatments; greater than 0, superiority of standard treatments. B) Meta-regression analysis. The results shows a significant trend toward HR = 1 (logHR = 0) over time in the GlaxoSmithKline (GSK) cohort. That is, the average effect size of treatments decreased over time [by 38% per decade ( = exp (0.48) = 1.62)] For the National Cancer Institute Canada Clinical Trials Group (CTG) cohort there was no statistically significant change in the treatment effect over time. R² = 24.48% for GSK i.e. the model accounted for about 25% of the observed variation in the results while R² was virtually zero in the CTG cohort. C) Meta-analysis stratified according to time periods. The results confirm the findings of meta-regression. Vertical lines indicate lines of no difference between new and standard treatments. Note that a “no difference” result can be obtained when treatments are truly identical, or when experimental treatments are as successful as standard treatments (i.e., sometimes new treatments are better and sometimes standard treatments are better). Squares indicate point estimates. Horizontal lines represent 99% confidence interval (CI).

Figure 11. Change in sample size over time.

There was an increase in the sample size over time both for National Cancer Institute Canada Clinical Trials Group (CTG) and GlaxoSmithKline (GSK) cohort. For CTG, on average, the sample size increased by 50% per decade, while for GSK, on average, it doubled. GSK trials were also somewhat larger than CTG trials: [median (range): 296 (20 to 8231) vs. 244 (31 to 5157); p = 0.062]. Sample size was somewhat larger in GSK trials [median (range): 296 (20 to 8231) compared to 244 (31 to 5157); p = 0.062]. It also increased over time in both cohorts. For CTG, on average the sample size increased by 50% per decade, while for GSK, on average, it doubled per decade. The average sample size increase (i.e. slope) was statistically significant between two cohorts (P<0.001).

Figure 12. Meta-regression of effect of time (year of publication), choice of active control and sample size on the magnitude effect in National Cancer Institute Canada Clinical Trials Group (CTG) cohort of trials (A) and GlaxoSmithKline (GSK) cohort (B).

None of the variables were statistically significant in NCIC CTG cohort of trials (R² = −0.68%). In GSK cohort sample size showed no statistical significant association with the results (p = 0.08) while year (p = 0.048) and the choice of comparator (p<0.000) were statistically associated with the observed results in GSK cohort. These two variable accounted for about 72% of the observed variation in the results (R² = 71.69%). In general, the effect size was closer to 1 (ln HR = 0) when the active comparator was employed.