Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2004;82(4):661-87.
doi: 10.1111/j.0887-378X.2004.00327.x.

Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages

Richard L Kravitz  1 , Naihua Duan, Joel Braslow
Affiliations

Evidence-based medicine, heterogeneity of treatment effects, and the trouble with averages

Richard L Kravitz et al. Milbank Q. 2004.

Erratum in

  • Milbank Q. 2006;84(4):759-60

Abstract

Evidence-based medicine is the application of scientific evidence to clinical practice. This article discusses the difficulties of applying global evidence ("average effects" measured as population means) to local problems (individual patients or groups who might depart from the population average). It argues that the benefit or harm of most treatments in clinical trials can be misleading and fail to reveal the potentially complex mixture of substantial benefits for some, little benefit for many, and harm for a few. Heterogeneity of treatment effects reflects patient diversity in risk of disease, responsiveness to treatment, vulnerability to adverse effects, and utility for different outcomes. Recognizing these factors, researchers can design studies that better characterize who will benefit from medical treatments, and clinicians and policymakers can make better use of the results.

PubMed Disclaimer

Figures

Figure 1
Figure 1. Distribution of Individual-Specific Treatment Effects (ITEs) in Three Hypothetical Populations
The average treatment effect (ATE) is indicated by the vertical bar. Individuals to the right of the bar derive a greater than average benefit, while those to the left derive less than an average benefit or even harm. The horizontal axis is standardized by the pooled standard deviation of the outcome. The gray zone represents patients in whom the effect size is so small (+/−0.25 SD) as to be clinically meaningless. Panel A represents a normal distribution centered on an effect size of 0.5 SD; panel B represents a more narrow distribution, still centered on the same mean; and panel C is normally distributed but shifted to the right (mean effect size about 1.3 SDs).
Figure 2
Figure 2. Distribution of ITE in the Population (Large Unshaded Curve) and in Three Hypothetical Samples (Shaded Curves)
Sample 1 is centered but fails to reflect the diversity of the population in terms of net treatment benefit. Sample 2 is composed of individuals who happen to derive much more net benefit from the treatment than does the average member of the population. Only sample 3 is broadly representative of the population in terms of risk, responsiveness, and vulnerability.
Figure 3
Figure 3
Relationships of Risk, Responsiveness, and Vulnerability to Treatment Thresholds
See this image and copyright information in PMC

References

    1. Ahles TA, Saykin AJ, Noll WW, Furstenberg CT, Guerin S, Cole B, Mott LA. The Relationship of APOE Genotype to Neuropsychological Performance in Long-Term Cancer Survivors Treated with Standard Dose Chemotherapy. Psycho-oncology. 2003;12:612–9. 10.1002/pon.742. - DOI - PubMed
    1. Allmark P. Should Research Samples Reflect the Diversity of the Population? Journal of Clinical Epidemiology. 2004;30:185–9. - PMC - PubMed
    1. B-Blocker Heart Attack Trial Research Group. A Randomized Trial of Propranolol in Patients with Acute Myocardial Infarction. I. Mortality Results. Journal of the American Medical Association. 1982;247:1707–14. 10.1001/jama.247.12.1707. - DOI - PubMed
    1. Baird KL. The New NIH and FDA Medical Research Policies: Targeting Gender, Promoting Justice. Journal of Health Politics, Policy and Law. 1999;24(3):531–65. - PubMed
    1. Brazell C, Freeman A, Mosteller M. Maximizing the Value of Medicines by Including Pharmacogenetic Research in Drug Development and Surveillance. British Journal of Clinical Pharmacology. 2002;53:224–31. - PMC - PubMed

Publication types