Fool's gold, lost treasures, and the randomized clinical trial

Abstract

Background: Randomized controlled trials with a survival endpoint are the gold standard for clinical research, but have failed to achieve cures for most advanced malignancies. The high costs of randomized clinical trials slow progress (thereby causing avoidable loss of life) and increase health care costs.

Discussion: A malignancy may be caused by several different mutations. Therapies effective vs one mutation may be discarded due to lack of statistical significance across the entire population. Conversely, expensive large randomized trials may have sufficient statistical power to demonstrate benefit despite the therapy only working in subgroups. Non-cost-effective therapy is then applied to all patients (including subgroups it cannot help). Randomized trials comparing therapies with different mechanisms of action are misleading since they may conclude the therapies are "equivalent" despite benefitting different subpopulations, or may erroneously conclude that one therapy is superior simply because it targets a larger subpopulation. Furthermore, minor variances in patient selection may determine study outcome, a therapy may be discarded as ineffective despite substantial benefit in one subpopulation if harmful in another, randomized trials may more effectively detect therapies with minor benefit in most patients vs marked benefit in subpopulations, and randomized trials in unselected patients may erroneously conclude that "shot-gun" combinations are superior to single agents when sequential administration of personalized single agents might work better and spare patients treatment with drugs that cannot help them. We must identify predictive biomarkers early by comparing responding to progressing patients in phase I-II trials. Enriching randomized trials for biomarker-positive patients can markedly reduce required patient numbers and costs despite expensive screening for biomarker-positive patients. Available data support approval of new drugs without randomized trials if they yield single-agent sustained responses in patients refractory to standard therapies. Conversely, new approaches are needed to guide development of drug combinations since both standard phase II approaches and phase II-III randomized trials have a high risk of misleading.

Summary: Traditional randomized clinical trials approaches are often inefficient, wasteful, and unreliable. New clinical research paradigms are needed. The primary outcome of clinical research should be "Who (if anyone) benefits?" rather than "Does the overall group benefit?"

Figure 1

Comparison of therapies hitting different simulated targets: Comparisons of a simulated therapy that quintupled survival in every 10^th patient starting with patient number 10 to another that quintupled survival in every 10^th patient starting with patient number 11 would erroneously conclude that the two therapies are equivalent (p=0.89), despite them being of benefit in completely different subpopulations.

Figure 2

Impact of minor changes in proportion of patients with target in simulated trials: If a new therapy quintupled survival in patients with a particular target, the 668-patient simulated study was negative if the target was present in 15% of patients (HR=0.81, p=0.06) but was positive if the target was present in just 11 more patients (16.7%) (HR=0.79, p=0.04). Hence, very minor variations in study patient populations can determine whether a trial will be negative vs positive.

Figure 3

Impact of benefit of erlotinib in one subpopulation vs harm in another: Despite substantial benefit in one subpopulation, a randomized trial may conclude that an agent is ineffective if it causes harm in a different subpopulation. Erlotinib vs placebo were added to chemotherapy in NSCLC, [33] and the curves overlapped suggesting no impact of erlotinib (two center curves, redrawn from Herbst et al. [33]). However, on molecular assessment, erlotinib was associated with potential benefit in the 13% of patients with an EGFR mutation (p=0.09), but was associated with harm in the 21% of patients with KRAS mutations (p=0.03) (curves resynthesized using component parts from Eberhard et al. [34]).

Figure 4

Therapy giving minor benefit in all patients achieved significance in simulated trial: In a 668 patient simulated study, a therapy that increased survival by 33% in all patients was judged to be effective (HR=0.80, p=0.03) (survival curves presented here), while a therapy that quintupled survival in 10% of the patients was judged ineffective (HR=0.85, p=0.16, see Figure 1 from Stewart, Whitney and Kurzrock [4]).

Figure 5

Suggested trial structure for assessment of combinations: Combination A+B would be compared to A followed at progression by B and to B followed at progression by A. Endpoints might include: 1) Time to progression on A+B vs time to progression from the initiation of the first single agent until completion of the second single agent; 2) A+B would be compared to each of A alone or B alone with respect to time to progression and maximum response; 3) Overall survival on the 3 arms; 4) Ability of B to suppress emergence of specific resistant clones while on A. 5) Of very high importance would be the development of molecular signatures that predict unique benefit of A alone, B alone, A+B combined, and the A→B/B→A sequences.