Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 Dec 26;8(12):e85035.
doi: 10.1371/journal.pone.0085035. eCollection 2013.

Using multiple types of studies in systematic reviews of health care interventions--a systematic review

Frank Peinemann  1 , Doreen Allen Tushabe  2 , Jos Kleijnen  3
Affiliations

Using multiple types of studies in systematic reviews of health care interventions--a systematic review

Frank Peinemann et al. PLoS One. .

Abstract

Background: A systematic review may evaluate different aspects of a health care intervention. To accommodate the evaluation of various research questions, the inclusion of more than one study design may be necessary. One aim of this study is to find and describe articles on methodological issues concerning the incorporation of multiple types of study designs in systematic reviews on health care interventions. Another aim is to evaluate methods studies that have assessed whether reported effects differ by study types.

Methods and findings: We searched PubMed, the Cochrane Database of Systematic Reviews, and the Cochrane Methodology Register on 31 March 2012 and identified 42 articles that reported on the integration of single or multiple study designs in systematic reviews. We summarized the contents of the articles qualitatively and assessed theoretical and empirical evidence. We found that many examples of reviews incorporating multiple types of studies exist and that every study design can serve a specific purpose. The clinical questions of a systematic review determine the types of design that are necessary or sufficient to provide the best possible answers. In a second independent search, we identified 49 studies, 31 systematic reviews and 18 trials that compared the effect sizes between randomized and nonrandomized controlled trials, which were statistically different in 35%, and not different in 53%. Twelve percent of studies reported both, different and non-different effect sizes.

Conclusions: Different study designs addressing the same question yielded varying results, with differences in about half of all examples. The risk of presenting uncertain results without knowing for sure the direction and magnitude of the effect holds true for both nonrandomized and randomized controlled trials. The integration of multiple study designs in systematic reviews is required if patients should be informed on the many facets of patient relevant issues of health care interventions.

PubMed Disclaimer

Conflict of interest statement

Competing Interests: No authors have any competing interests. Employment by Kleijnen Systematic Reviews Ltd does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.

Figures

Figure 1
Figure 1. Literature retrieval and study selection.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Haynes B (1999) Can it work? Does it work? Is it worth it? The testing of health care interventions is evolving. BMJ 319: 652-653. doi:10.1136/bmj.319.7211.652. PubMed: 10480802. - DOI - PMC - PubMed
    1. Djulbegovic B, Paul A (2011) From efficacy to effectiveness in the face of uncertainty: indication creep and prevention creep. JAMA 305: 2005-2006. PubMed: 21586716. - PubMed
    1. Victora CG, Habicht JP, Bryce J (2004) Evidence-based public health: moving beyond randomized trials. Am J Public Health 94: 400-405. doi:10.2105/AJPH.94.3.400. PubMed: 14998803. - DOI - PMC - PubMed
    1. Djulbegovic M, Djulbegovic B (2011) Implications of the principle of question propagation for comparative-effectiveness and "data mining" research. JAMA 305: 298-299. doi:10.1001/jama.2010.2013. PubMed: 21245185. - DOI - PubMed
    1. Vandenbroucke JP (2008) Observational research, randomised trials, and two views of medical science. PLoS Med 5: e67. doi:10.1371/journal.pmed.0050067. PubMed: 18336067. - DOI - PMC - PubMed

Publication types