Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Comparative Study
. 2017 Feb 15;6(1):28.
doi: 10.1186/s13643-017-0422-x.

Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project

Jeffrey Low  1 , Joseph S Ross  2   3   4   5 , Jessica D Ritchie  2 , Cary P Gross  3   4 , Richard Lehman  6 , Haiqun Lin  2   7 , Rongwei Fu  8 , Lesley A Stewart  9 , Harlan M Krumholz  10   11   12   13
Affiliations
Comparative Study

Comparison of two independent systematic reviews of trials of recombinant human bone morphogenetic protein-2 (rhBMP-2): the Yale Open Data Access Medtronic Project

Jeffrey Low et al. Syst Rev. .

Abstract

Background: It is uncertain whether the replication of systematic reviews, particularly those with the same objectives and resources, would employ similar methods and/or arrive at identical findings. We compared the results and conclusions of two concurrent systematic reviews undertaken by two independent research teams provided with the same objectives, resources, and individual participant-level data.

Methods: Two centers in the USA and UK were each provided with participant-level data on 17 multi-site clinical trials of recombinant human bone morphogenetic protein-2 (rhBMP-2). The teams were blinded to each other's methods and findings until after publication. We conducted a retrospective structured comparison of the results of the two systematic reviews. The main outcome measures included (1) trial inclusion criteria; (2) statistical methods; (3) summary efficacy and risk estimates; and (4) conclusions.

Results: The two research teams' meta-analyses inclusion criteria were broadly similar but differed slightly in trial inclusion and research methodology. They obtained similar results in summary estimates of most clinical outcomes and adverse events. Center A incorporated all trials into summary estimates of efficacy and harms, while Center B concentrated on analyses stratified by surgical approach. Center A found a statistically significant, but small, benefit whereas Center B reported no advantage. In the analysis of harms, neither showed an increased cancer risk at 48 months, although Center B reported a significant increase at 24 months. Conclusions reflected these differences in summary estimates of benefit balanced with small but potentially important risk of harm.

Conclusions: Two independent groups given the same research objectives, data, resources, funding, and time produced broad general agreement but differed in several areas. These differences, the importance of which is debatable, indicate the value of the availability of data to allow for more than a single approach and a single interpretation of the data.

Systematic review registration: PROSPERO CRD42012002040 and CRD42012001907 .

Keywords: Data interpretation; Data sharing; Meta-analysis; Systematic review.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
YODA Project timeline for the independent synthesis and meta-analysis of rhBMP-2 clinical trials, including their publication
Fig. 2
Fig. 2
Forest plots from Center A and Center B meta-analyses examining likelihood of bone fusion and cancer risk associated with rhBMP-2
See this image and copyright information in PMC

References

    1. Ioannidis JP. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005;294(2):218–228. doi: 10.1001/jama.294.2.218. - DOI - PubMed
    1. Stewart LA, Tierney JF. To IPD or not to IPD? Advantages and disadvantages of systematic reviews using individual patient data. Eval Health Prof. 2002;25(1):76–97. doi: 10.1177/0163278702025001006. - DOI - PubMed
    1. Wang JG, Staessen JA, Franklin SS, Fagard R, Gueyffier F. Systolic and diastolic blood pressure lowering as determinants of cardiovascular outcome. Hypertension. 2005;45(5):907–913. doi: 10.1161/01.HYP.0000165020.14745.79. - DOI - PubMed
    1. Stewart LA, Parmar MK. Meta-analysis of the literature or of individual patient data: is there a difference? Lancet. 1993;341(8842):418–422. doi: 10.1016/0140-6736(93)93004-K. - DOI - PubMed
    1. Riley RD, Lambert PC, Staessen JA, Wang J, Gueyffier F, Thijs L, Boutitie F. Meta-analysis of continuous outcomes combining individual patient data and aggregate data. Stat Med. 2008;27(11):1870–1893. doi: 10.1002/sim.3165. - DOI - PubMed

Publication types

MeSH terms

Substances

LinkOut - more resources