Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Aug;26(4):364-368.
doi: 10.1017/S2045796016000809. Epub 2016 Oct 28.

Pre-post effect sizes should be avoided in meta-analyses

P Cuijpers  1 , E Weitz  1 , I A Cristea  2 , J Twisk  3
Affiliations

Pre-post effect sizes should be avoided in meta-analyses

P Cuijpers et al. Epidemiol Psychiatr Sci. 2017 Aug.

Abstract

Aims The standardised mean difference (SMD) is one of the most used effect sizes to indicate the effects of treatments. It indicates the difference between a treatment and comparison group after treatment has ended, in terms of standard deviations. Some meta-analyses, including several highly cited and influential ones, use the pre-post SMD, indicating the difference between baseline and post-test within one (treatment group).

Methods: In this paper, we argue that these pre-post SMDs should be avoided in meta-analyses and we describe the arguments why pre-post SMDs can result in biased outcomes.

Results: One important reason why pre-post SMDs should be avoided is that the scores on baseline and post-test are not independent of each other. The value for the correlation should be used in the calculation of the SMD, while this value is typically not known. We used data from an 'individual patient data' meta-analysis of trials comparing cognitive behaviour therapy and anti-depressive medication, to show that this problem can lead to considerable errors in the estimation of the SMDs. Another even more important reason why pre-post SMDs should be avoided in meta-analyses is that they are influenced by natural processes and characteristics of the patients and settings, and these cannot be discerned from the effects of the intervention. Between-group SMDs are much better because they control for such variables and these variables only affect the between group SMD when they are related to the effects of the intervention.

Conclusions: We conclude that pre-post SMDs should be avoided in meta-analyses as using them probably results in biased outcomes.

Keywords: Individual patient data; meta-analyses; pre-post effect sizes; standardised mean difference.

PubMed Disclaimer

References

    1. Balk EM, Earley A, Patel K, Trikalinos TA, Dahabreh IJ (2012). Empirical Assessment of Within-arm Correlation Imputation in Trials of Continuous Outcomes (Methods Research Report No. AHRQ Publication No. 12(13)-EHC141-EF). Agency for Healthcare Research and Quality: Rockville, MD. Retrieved from www.effectivehealthcare.ahrq.gov/reports/final.cfm - PubMed
    1. Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J (1961). An inventory for measuring depression. Archives of General Psychiatry 4, 561–571. - PubMed
    1. Beck AT, Steer RA, Brown GK (1996). BDI-II. Beck Depression Inventory, 2nd edn. Manual. Psychological Corporation: San Antonio.
    1. Biostat Inc (2015). Comprehensive Meta-Analysis (Version 2.2.064). Retrieved from www.meta-analysis.com
    1. Cuijpers P, Karyotaki E, Weitz E, Andersson G, Hollon SD, van Straten A (2014a). The effects of psychotherapies for major depression in adults on remission, recovery and improvement: a meta-analysis. Journal of Affective Disorders 159, 118–126. - PubMed

Publication types

MeSH terms

Substances