Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2010 Jul 16:341:c3515.
doi: 10.1136/bmj.c3515.

Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study

Eveline Nüesch  1 , Sven Trelle, Stephan Reichenbach, Anne W S Rutjes, Beatrice Tschannen, Douglas G Altman, Matthias Egger, Peter Jüni
Affiliations
Meta-Analysis

Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study

Eveline Nüesch et al. BMJ. .

Abstract

Objective: To examine the presence and extent of small study effects in clinical osteoarthritis research.

Design: Meta-epidemiological study.

Data sources: 13 meta-analyses including 153 randomised trials (41 605 patients) that compared therapeutic interventions with placebo or non-intervention control in patients with osteoarthritis of the hip or knee and used patients' reported pain as an outcome.

Methods: We compared estimated benefits of treatment between large trials (at least 100 patients per arm) and small trials, explored funnel plots supplemented with lines of predicted effects and contours of significance, and used three approaches to estimate treatment effects: meta-analyses including all trials irrespective of sample size, meta-analyses restricted to large trials, and treatment effects predicted for large trials.

Results: On average, treatment effects were more beneficial in small than in large trials (difference in effect sizes -0.21, 95% confidence interval -0.34 to -0.08, P=0.001). Depending on criteria used, six to eight funnel plots indicated small study effects. In six of 13 meta-analyses, the overall pooled estimate suggested a clinically relevant, significant benefit of treatment, whereas analyses restricted to large trials and predicted effects in large trials yielded smaller non-significant estimates.

Conclusions: Small study effects can often distort results of meta-analyses. The influence of small trials on estimated treatment effects should be routinely assessed.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that all authors had: (1) No financial support for the submitted work from anyone other than their employer; (2) No financial relationships with commercial entities that might have an interest in the submitted work; (3) No spouses, partners, or children with relationships with commercial entities that might have an interest in the submitted work; (4) No Non-financial interests that may be relevant to the submitted work.

Figures

None
Fig 1 Difference in effect sizes between 95 small trials with fewer than 100 patients per arm and 58 large trials. Negative differences indicate that small trials show more beneficial treatment effects. P values are for interaction between sample size and effect sizes. NSAIDs=non-steroidal anti-inflammatory drugs
None
Fig 2 Funnel plots of 13 included meta-analyses including prediction lines from univariable meta-regression models with SE as explanatory variable (dashed red) and 5% contour areas to display areas of significance (blue) and non-significance (pale blue). Lines for predicted effects should be interpreted independently of contours delineated by shaded areas. Extent of deviation of lines for predicted effects from vertical line should be considered, irrespective of relation of lines to contours. P values derived from regression tests for asymmetry. NSAIDs=non-steroidal anti-inflammatory drugs
None
Fig 3 Results of individual random effects meta-analyses of all trials (blue circle), results of random effects meta-analyses restricted to large trials with at least 100 patients per arm (open circle), and effect sizes for trials with SE of 0.1 predicted from random effects meta-regression models (green square). NSAIDs=non-steroidal anti-inflammatory drugs
See this image and copyright information in PMC

Comment in

References

    1. Sterne JA, Egger M, Smith GD. Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ 2001;323:101-5. - PMC - PubMed
    1. Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 2000;53:1119-29. - PubMed
    1. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 2005;330:753. - PMC - PubMed
    1. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457-65. - PubMed
    1. Egger M, Smith GD. Bias in location and selection of studies. BMJ 1998;316:61-6. - PMC - PubMed

Publication types

MeSH terms