Avoidance of large biases and large random errors in the assessment of moderate treatment effects: the need for systematic overviews

Abstract

In order to avoid selective biases and to minimize random errors, inference about the effects of treatment on serious endpoints needs to be based not on one, or a few, of the available trial results, but on a systematic overview of the totality of the evidence from all the relevant unconfounded randomized trials. But, only where coverage of all, or nearly all, randomized patients in all relevant trials (or a reasonably unbiased sample of such trials) can be assured, is a systematic overview of trials reasonably trustworthy, for then any selective biases are likely to be small in comparison with any moderate effects of treatment. Checks for the existence of such biases can best be conducted if reasonably detailed data are available from each trial. Future trials should take into account the results of any relevant overviews in their design, and should plan to obtain sufficient numbers of events to contribute substantially to such overviews. In many cases, this implies the need for randomized trials that are much larger than is currently standard.