Flawed meta-analyses comparing psychotherapy with pharmacotherapy

Abstract

Objective: The author sought to illustrate the invalidity of meta-analyses that claim to quantitatively compare the benefits of psychotherapy to pharmacotherapy in patients with psychiatric disorders.

Method: Studies included in four meta-analyses were retrieved and their study designs evaluated.

Results: The meta-analyses compared effect sizes from disparate studies that were not uniformly blind, random, controlled, or of high quality. The studies did not directly address the comparative efficacy question or lacked assay sensitivity.

Conclusions: Numerous types of studies exemplify the need for caution in evaluating meta-analytic conclusions without first critically examining the included studies. Estimates of the relative efficacy of different treatments are not well founded when based almost exclusively on indirect, multiply confounded comparisons. Meta-analyses based on flawed studies or studies that lack demonstrated assay sensitivity are also inadequate for the criticism of treatment guidelines. Some bodies of data are inadequate to support a proper meta-analysis.