Measuring effects without randomized trials? Options, problems, challenges

Abstract

The idea of using information routinely generated in the course of health care delivery for assessing the relative efficacy of alternative therapies has an undeniable attraction. If feasible, it would enable the difficulties of running a randomized controlled trial (RCT) to be bypassed. But grave obstacles, both practical and theoretical, beset the effort. If, nonetheless, the data base approach is chosen, certain suggestions may be helpful. First, make sure that necessity actually prevents using an RCT. Reconsider that question. Special attention is due to two variants of the usual RCT: the "firms" approach and the large simple trial. If, after reconsideration, a data base approach still is chosen, then let the undertaking be carefully planned, with participation from those who will produce the data and also from future users of the results. Let the planning result in a protocol, and in provisions for a quality assurance program. Finally, it is time to respond to a challenge left us by David Byar: arrange to record why the patient is being given the therapy selected. This information should be a powerful adjustment variable; arranging to collect it will call for imaginative thinking, experimentation, and patience, but it is an idea deserving much effort.