Evolution of treatment effects over time: empirical insight from recursive cumulative metaanalyses

Abstract

Evidence on how much medical interventions work may change over time. It is important to determine what fluctuations in the treatment effect reported by randomized trials and their metaanalyses may be expected and whether extreme fluctuations signal future major changes. We applied recursive cumulative metaanalysis of randomized controlled trials to evaluate the relative change in the pooled treatment effect (odds ratio) over time for 60 interventions in two medical fields (pregnancy/perinatal medicine, n = 45 interventions; myocardial infarction, n = 15 interventions). We evaluated the scatter of relative changes for different numbers of total patients in previous trials. Outlier cases were noted with changes greater than 2.5 standard deviations of the expected. With 500 accumulated patients, the pooled odds ratio may change by 0.6- to 1.7-fold in the immediate future. When 2000 patients have already been randomized, the respective figures are between 0.74- and 1.35-fold for pregnancy/perinatal medicine and between 0.83- and 1.21-fold for myocardial infarction studies. Extreme early fluctuations in the treatment effect were observed in three interventions (magnesium in myocardial infarction, calcium and antiplatelet agents for prevention of preeclampsia), where recent mega-trials have contradicted prior metaanalyses, as well as in four other examples where early large treatment effects were dissipated when more data appeared. Past experience may help quantify the uncertainty surrounding the treatment effects reported in early clinical trials and their metaanalyses. Early wide oscillations in the evolution of the treatment effect for specific interventions may sometimes signal further major changes in the future.

Figure 1

Illustrative example of a standard metaanalysis on calcium supplementation for prevention of preeclampsia (Left), the respective cumulative metaanalysis (Center), and the respective recursive cumulative metaanalysis (Right). Cumulative metaanalysis is performed at the end of each year when new trials have been published in this year. The odds ratio of the cumulative metaanalysis at the end of each year is displayed according to the cumulative number of patients. The relative change in the odds ratio is also displayed according to the same scale. Calculations are based on random effects. PTs, patients; Cum Pts, cumulative patients; CI, confidence interval.

Figure 2

Scatter plots of the relative change in the pooled odds ratio as a function of the cumulative number of patients in previously published randomized trials. An outlier relative change of 0.05 for sample size = 50 is not shown in the myocardial infarction graph. A relative change greater than 1 means that the odds ratio tends to increase with the accumulation of more data. Fixed effects graphs were similar.

Figure 3

Recursive cumulative metaanalyses for 10 perinatal topics (Upper) and 10 myocardial infarction topics (Lower). The controversial cases of antiplatelet agents (bold continuous line) and calcium (bold interrupted line) used to prevent preeclampsia during pregnancy and magnesium salts (bold continuous line) and nitrates (bold interrupted line) to decrease mortality in acute myocardial infarction (AMI) clearly show larger oscillations in the pooled treatment effect over time than the other topics. Recent large trials for these four topics have been included. Other topics depicted in the pregnancy/perinatal interventions panel include balanced protein/energy supplementation in pregnancy, extended spectrum vs. first-generation cephalosporins with cesarean section, prophylactic administration of synthetic surfactant, birthing chair vs. recumbent position for second stage of labor, prophylactic phenobarbital in very-low-birth-weight neonates, prophylactic indomethacin in preterm infants, prophylactic oral betamimetics in pregnancy, and corticosteroids after preterm prelabor rupture of membranes. Other topics depicted in the myocardial infarction interventions panel include beta-blockers in AMI, beta-blockers for secondary prevention, calcium channel blockers for AMI, calcium channel blockers for secondary prevention, oral anticoagulants for AMI, class I antiarrhythmics in AMI, prophylactic lidocaine in AMI, and i.v. streptokinase in AMI. Calculations are based on fixed effects, but random-effects graphs are very similar. A relative change greater than 1 means that the odds ratio tends to increase with the accumulation of more data.