External validity: the neglected dimension in evidence ranking

Abstract

Evidence that is both accurate (internally valid) and relevant (externally valid) is needed to decide which treatment is best for a particular patient. Evidence rankings facilitate the marshalling of evidence on clinical decisions in the common context of an overwhelming number of studies, some with conflicting results. Evidence from randomized control trials is typically ranked above evidence from non-experimental studies since rankings are based primarily, if not exclusively, on considerations of internal validity. We propose that evidence rankings should consider equally both internal and external validity. External validity includes how closely the study population, the institution types in the study, the types of physicians in the study, the role of clinician decision-making (e.g. dose adjustment) in the study, and the role of patient preferences in the study resemble those in actual practice. The example of spironolactone use in heart failure illustrates the danger in using evidence that is internally but not externally valid. Ideally, a treatment should only be used when both internally and externally valid evidence indicates that it will be useful for the particular patient.