Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017;113(1):651-671.
doi: 10.1007/s11192-017-2375-1. Epub 2017 Apr 3.

Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication

Michail Kovanis  1   2 , Ludovic Trinquart  1   3 , Philippe Ravaud  1   2   4   3   5 , Raphaël Porcher  1   2   4
Affiliations

Evaluating alternative systems of peer review: a large-scale agent-based modelling approach to scientific publication

Michail Kovanis et al. Scientometrics. 2017.

Abstract

The debate on whether the peer-review system is in crisis has been heated recently. A variety of alternative systems have been proposed to improve the system and make it sustainable. However, we lack sufficient evidence and data related to these issues. Here we used a previously developed agent-based model of the scientific publication and peer-review system calibrated with empirical data to compare the efficiency of five alternative peer-review systems with the conventional system. We modelled two systems of immediate publication, with and without online reviews (crowdsourcing), a system with only one round of reviews and revisions allowed (re-review opt-out) and two review-sharing systems in which rejected manuscripts are resubmitted along with their past reviews to any other journal (portable) or to only those of the same publisher but of lower impact factor (cascade). The review-sharing systems outperformed or matched the performance of the conventional one in all peer-review efficiency, reviewer effort and scientific dissemination metrics we used. The systems especially showed a large decrease in total time of the peer-review process and total time devoted by reviewers to complete all reports in a year. The two systems with immediate publication released more scientific information than the conventional one but provided almost no other benefit. Re-review opt-out decreased the time reviewers devoted to peer review but had lower performance on screening papers that should not be published and relative increase in intrinsic quality of papers due to peer review than the conventional system. Sensitivity analyses showed consistent findings to those from our main simulations. We recommend prioritizing a system of review-sharing to create a sustainable scientific publication and peer-review system.

Keywords: Agent-based model; Cascade; Complex systems; Peer review; Portable; Post-publication.

PubMed Disclaimer

Figures

Fig. 1
Fig. 1
Diagrams of the alternative peer-review systems
See this image and copyright information in PMC

References

    1. Allesina S. Modeling peer review: An agent-based approach. Ideas in Ecology and Evolution. 2012;5(2):27–35. doi: 10.4033/iee.2012.5b.8.f. - DOI
    1. Arns M. Open access is tiring out peer reviewers. Nature. 2014;515(7528):467. doi: 10.1038/515467a. - DOI - PubMed
    1. Bianchi F, Squazzoni F. Is three better than one? Simulating the effect of reviewer selection and behavior on the quality and efficiency of peer review. In: Yilmaz L, Chan WKV, Moon I, Roeder TMK, Macal C, Rossetti MD, editors. Proceedings of the 2015 Winter Simulation Conference. NJ: IEEE Press, Piscatawaw; 2015. pp. 4081–4089.
    1. Bohannon J. Who’s afraid of peer review? Science. 2013;342(6154):60–65. doi: 10.1126/science.342.6154.60. - DOI - PubMed
    1. Bonabeau E. Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences. 2002;99(suppl 3):7280–7287. doi: 10.1073/pnas.082080899. - DOI - PMC - PubMed

LinkOut - more resources