Threats to the validity of clinical trials employing enrichment strategies for sample selection

Abstract

Subject selection and exclusion criteria employed in typical clinical effectiveness trials of investigational new drugs have two fundamental aims: (1) to ensure that patients entering a study are truly suffering from the condition the drug is intended to treat and (2) to maximize the likelihood that the study will detect an effect of the drug if, in fact, one exists. Typical protocol selection criteria not only specify exacting procedures for establishing and documenting the diagnosis of those recruited for a study but also seek to increase, relative to the prevalence in the general population, the proportion of individuals in the sample likely to respond to pharmacological treatment. Because it is ordinarily impossible to learn prior to extensive clinical experience with a new drug which, if any, patient characteristics reliably predict a consistent treatment response, strategies for sample "enrichment" typically operate by excluding patients (for example, those with very advanced and/or complicated illness, those with serious concomitant illness, those at the extremes of age, those with very mild illness, and so forth) in whom a dependable response to treatment seems unlikely on logical and/or generic grounds. Some studies use positive strategies for sample "enrichment." In studies evaluating drugs intended to treat recurrent episodes of psychiatric illnesses, many protocols recommend selective recruitment of patients with a history of meaningful positive responses to antipsychotic treatment during prior episodes. Sample selection procedures of these kinds impose limits on the generalizability of a study's results (i.e., external validity), but the use of nonrandom patient samples is ordinarily held to have no effect on the internal validity of the results. In short, studies employing highly selected patient samples are, despite their limited external validity, regularly accepted as valid sources of evidence bearing on a drug's effectiveness. There are exceptions, however; this paper describes one in which the use of a seemingly innocuous sample enrichment maneuver proved highly damaging to the ultimate credibility of an important multicenter trial. In particular, exposure to an experimental treatment during an open qualification phase may invalidate drug-placebo comparisons made during a later randomized, blinded, controlled phase. Our review of the trial also reveals that the enrichment maneuver employed probably failed to accomplish its intended aims, selecting patients whose improvements on the outcome variable may be as reasonably ascribed to chance as to drug effect. This is all the more surprising because the method of sample enrichment employed has much in common with those long recommended in the clinical trial literature.