Review

. 2018 Feb 13:360:k400.

doi: 10.1136/bmj.k400.

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine

Florian Naudet¹, Charlotte Sakarovitch², Perrine Janiaud¹, Ioana Cristea^{1

3}, Daniele Fanelli^{1

4}, David Moher^{1

5}, John P A Ioannidis^{6

7}

Affiliations

¹ Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA.
² Quantitative Sciences Unit, Division of Biomedical Informatics Research, Department of Medicine, Stanford University, Stanford, CA, USA.
³ Department of Clinical Psychology and Psychotherapy, Babes-Bolyai University, Romania.
⁴ Department of Methodology, London School of Economics and Political Science, UK.
⁵ Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada.
⁶ Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA jioannid@stanford.edu.
⁷ Departments of Medicine, of Health Research and Policy, of Biomedical Data Science, and of Statistics, Stanford University, Stanford, California, USA.

PMID: 29440066
PMCID: PMC5809812
DOI: 10.1136/bmj.k400

Review

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine

Florian Naudet et al. BMJ. 2018.

. 2018 Feb 13:360:k400.

doi: 10.1136/bmj.k400.

Authors

Florian Naudet¹, Charlotte Sakarovitch², Perrine Janiaud¹, Ioana Cristea^{1

3}, Daniele Fanelli^{1

4}, David Moher^{1

5}, John P A Ioannidis^{6

7}

Affiliations

¹ Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA.
² Quantitative Sciences Unit, Division of Biomedical Informatics Research, Department of Medicine, Stanford University, Stanford, CA, USA.
³ Department of Clinical Psychology and Psychotherapy, Babes-Bolyai University, Romania.
⁴ Department of Methodology, London School of Economics and Political Science, UK.
⁵ Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, ON, Canada.
⁶ Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, USA jioannid@stanford.edu.
⁷ Departments of Medicine, of Health Research and Policy, of Biomedical Data Science, and of Statistics, Stanford University, Stanford, California, USA.

PMID: 29440066
PMCID: PMC5809812
DOI: 10.1136/bmj.k400

Abstract

Objectives: To explore the effectiveness of data sharing by randomized controlled trials (RCTs) in journals with a full data sharing policy and to describe potential difficulties encountered in the process of performing reanalyses of the primary outcomes.

Design: Survey of published RCTs.

Setting: PubMed/Medline.

Eligibility criteria: RCTs that had been submitted and published by The BMJ and PLOS Medicine subsequent to the adoption of data sharing policies by these journals.

Main outcome measure: The primary outcome was data availability, defined as the eventual receipt of complete data with clear labelling. Primary outcomes were reanalyzed to assess to what extent studies were reproduced. Difficulties encountered were described.

Results: 37 RCTs (21 from The BMJ and 16 from PLOS Medicine) published between 2013 and 2016 met the eligibility criteria. 17/37 (46%, 95% confidence interval 30% to 62%) satisfied the definition of data availability and 14 of the 17 (82%, 59% to 94%) were fully reproduced on all their primary outcomes. Of the remaining RCTs, errors were identified in two but reached similar conclusions and one paper did not provide enough information in the Methods section to reproduce the analyses. Difficulties identified included problems in contacting corresponding authors and lack of resources on their behalf in preparing the datasets. In addition, there was a range of different data sharing practices across study groups.

Conclusions: Data availability was not optimal in two journals with a strong policy for data sharing. When investigators shared data, most reanalyses largely reproduced the original results. Data sharing practices need to become more widespread and streamlined to allow meaningful reanalyses and reuse of data.

Trial registration: Open Science Framework osf.io/c4zke.

Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

PubMed Disclaimer

Conflict of interest statement

Competing interests: All authors have completed the ICMJE uniform disclosure form at http://www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare that (1) No authors have support from any company for the submitted work; (2) FN has relationships (travel/accommodations expenses covered/reimbursed) with Servier, BMS, Lundbeck, and Janssen who might have an interest in the work submitted in the previous three years. In the past three years PJ received a fellowship/grant from GSK for her PhD as part of a public-private collaboration. CS, IC, DF, DM, and JPAI have no relationship with any company that might have an interest in the work submitted; (3) no author’s spouse, partner, or children have any financial relationships that could be relevant to the submitted work; and (4) none of the authors has any non-financial interests that could be relevant to the submitted work.

Figures

**Fig 2**
P values in initial analyses and in reanalyses. Axes are on a log scale. Blue indicates identical conclusion between initial analysis and reanalysis. Dots of same colors indicate analyses from same study

See this image and copyright information in PMC

Comment in

Data sharing in medical research.
Packer M. Packer M. BMJ. 2018 Feb 14;360:k510. doi: 10.1136/bmj.k510. BMJ. 2018. PMID: 29444885 No abstract available.

References

1. Taichman DB, Backus J, Baethge C, et al. Sharing Clinical Trial Data: A Proposal from the International Committee of Medical Journal Editors. PLoS Med 2016;13:e1001950. 10.1371/journal.pmed.1001950. - DOI - PMC - PubMed
1. Platt R, Ramsberg J. Challenges for Sharing Data from Embedded Research. N Engl J Med 2016;374:1897. 10.1056/NEJMc1602016. - DOI - PubMed
1. Kalager M, Adami HO, Bretthauer M. Recognizing Data Generation. N Engl J Med 2016;374:1898. 10.1056/NEJMc1603789. - DOI - PubMed
1. Devereaux PJ, Guyatt G, Gerstein H, Connolly S, Yusuf S, International Consortium of Investigators for Fairness in Trial Data Sharing Toward Fairness in Data Sharing. N Engl J Med 2016;375:405-7. 10.1056/NEJMp1605654. - DOI - PubMed
1. Haug CJ. Whose Data Are They Anyway? Can a Patient Perspective Advance the Data-Sharing Debate? N Engl J Med 2017;376:2203-5. 10.1056/NEJMp1704485. - DOI - PubMed

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations
Molecular Biology Databases
- NIAID Data Ecosystem - Find datasets on Infectious and Immune-mediated Diseases
Miscellaneous
- NCI CPTAC Assay Portal

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine

Affiliations

Data sharing and reanalysis of randomized controlled trials in leading biomedical journals with a full data sharing policy: survey of studies published in The BMJ and PLOS Medicine

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

Comment in

References

Publication types

MeSH terms

LinkOut - more resources

Full Text Sources

Other Literature Sources

Molecular Biology Databases

Miscellaneous