Measuring effectiveness. What to expect without a randomized control group

Abstract

Randomized controlled trials or studies are often considered the ideal way to evaluate the effectiveness of a treatment compared to a control. In such a study, the randomization procedure ensures that the subjects receiving the treatment and control are equal with respect to all conditions except for receiving the treatment or the control. Differences found by statistical comparisons of the results of such a study can be attributed to the effect of the treatment or how much the treatment differs from the control when all other things are held constant. Randomized controlled trials are not always possible, and even when possible they are often performed with such restrictions that they do not provide the true measure of the effectiveness of the treatment in the "real world" or under "conditions of usual practice." This article reviews the use of nonrandomized studies to measure effectiveness when a randomized control group is not available. Various types of nonrandomized studies are reviewed, along with their advantages and disadvantages. Often, these studies require statistical adjustments such as matching or covariance analysis to adjust for inequalities or to remove biases between the treatment and control groups; these are reviewed as well.