Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Dec 14;95(1):2-7.
doi: 10.1136/jnnp-2023-331941.

Comparison of relative change with effect size metrics in Alzheimer's disease clinical trials

Terry E Goldberg  1 , Seonjoo Lee  2   3 , Davangere P Devanand  4 , Lon S Schneider  5
Affiliations

Comparison of relative change with effect size metrics in Alzheimer's disease clinical trials

Terry E Goldberg et al. J Neurol Neurosurg Psychiatry. .

Abstract

Background: Per cent slowing of decline is frequently used as a metric of outcome in Alzheimer's disease (AD) clinical trials, but it may be misleading. Our objective was to determine whether per cent slowing of decline or Cohen's d is the more valid and informative measure of efficacy.

Methods: Outcome measures of interest were per cent slowing of decline; Cohen's d effect size and number-needed-to-treat (NNT). Data from a graphic were used to model the inter-relationships among Cohen's d, placebo decline in raw score units and per cent slowing of decline with active treatment. NNTs were computed based on different magnitudes of d. Last, we tabulated recent AD anti-amyloid clinical trials that reported per cent slowing and for which we computed their respective d's and NNTs.

Results: We demonstrated that d and per cent slowing were potentially independent. While per cent slowing of decline was dependent on placebo decline and did not include variance in its computation, d was dependent on both group mean difference and pooled SD. We next showed that d was a critical determinant of NNT, such that NNT was uniformly smaller when d was larger. In recent AD associated trials including those focused on anti-amyloid biologics, d's were below 0.23 and thus considered small, while per cent slowing was in the 22-29% range and NNTs ranged from 14 to 18.

Conclusions: Standardised effect size is a more meaningful outcome than per cent slowing of decline because it determines group overlap, which can directly influence NNT computations, and yield information on the likelihood of minimum clinically important differences. In AD, greater use of effect sizes, NNTs, rather than relative per cent slowing, will improve the ability to interpret clinical trial results and evaluate the clinical meaningfulness of statistically significant results.

Keywords: ALZHEIMER'S DISEASE; COGNITION; DEMENTIA; STATISTICS.

PubMed Disclaimer

Conflict of interest statement

Competing interests: LSS reports personal fees from AC Immune, Alpha-cognition, Athira, Corium, Cortexyme, BioVie, Eli Lilly, GW Research, Lundbeck, Merck, Neurim, Novo-Nordisk, Otsuka, Roche/Genentech, Cognition Therapeutics, Takeda; grants from Biohaven, Biogen, Eisai, Eli Lilly and Novartis. DPD reports research support from the National Institute on Aging, Alzheimer’s Association, is a scientific adviser to Acadia, TauRx, Corium, Genentech, and is a member of the Data and Safety Monitoring Board of BioXcel. TG and SL have no conflicts of interest to disclose.

Figures

Figure 1.
Figure 1.
The figure has 3 axes: Cohen’s d, percent slowing of decline, and placebo change in raw scores. For any given d (curved lines) holding the difference between groups constant and varying the pooled SDs, yielded a wide range of percent slowing values. Two examples make this clear. For 25 percent slowing of decline (and a placebo change of 2 units) multiple Cohens d exist as shown in the blue vertical line. See also the second blue line representing a 50 percent slowing of decline. Conversely, for a given Cohen’s d=.25, percent slowing of decline can range from 0 to near 100 percent, as based on placebo change in raw scores.
Figure 2.
Figure 2.
NNTs examined as a function of Cohen’s d and event rate in the placebo group. Here we examined NNT at multiple d’s and event rates. CER (control event rate, i.e., response rate in the placebo group) represents the proportion of advantageous outcomes in the placebo group. For any given CER, the larger the d, the smaller the NNT.
See this image and copyright information in PMC

Comment in

  • How effective is effective enough?
    Kalincik T, Brodtmann A. Kalincik T, et al. J Neurol Neurosurg Psychiatry. 2023 Dec 14;95(1):1. doi: 10.1136/jnnp-2023-332311. J Neurol Neurosurg Psychiatry. 2023. PMID: 37989567 No abstract available.

References

    1. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed: Lawrence Erlbaum Associates; 1988.
    1. Furukawa TA. From effect size into number needed to treat. Lancet. 1999;353(9165):1680. - PubMed
    1. Furukawa TA, Leucht S. How to obtain NNT from Cohen's d: comparison of two methods. PLoS One. 2011;6(4):e19070. - PMC - PubMed
    1. van Dyck CH, Swanson CJ, Aisen P, Bateman RJ, Chen C, Gee M, et al. Lecanemab in Early Alzheimer's Disease. N Engl J Med. 2023;388(1):9–21. - PubMed
    1. Magnusson K. Interpreting Cohen’s d Effect Size: An interactive visualization R Psychologist 2022. [updated September 19, 2022. Available from: https://rpsychologist.com/cohend/.

Publication types