Zero effects in substance abuse programs. Avoiding false positives and false negatives in the evaluation of community-based programs

Abstract

The evaluation of community-based programs poses special design and analysis problems. The present article focuses on two major types of errors that can occur in such evaluations: false positive--incorrectly declaring a program to be effective--and false negatives--incorrectly declaring a program to be ineffective. The evaluation of a national demonstration of community-based programs to reduce substance abuse, Fighting Back, is used to illustrate several approaches to reduce the probability of errors. Both those errors that are affected by the design and those by analytic approaches are considered. Ways to assess multiple outcomes and to match the complexity of the program with design and analytic strategies are proposed. Community trials are complex interventions, and, although they can provide very useful information, their outcomes have to be understood in terms of the constructs they test and the contexts within which they are carried out.