The Global Burden of Journal Peer Review in the Biomedical Literature: Strong Imbalance in the Collective Enterprise

Abstract

The growth in scientific production may threaten the capacity for the scientific community to handle the ever-increasing demand for peer review of scientific publications. There is little evidence regarding the sustainability of the peer-review system and how the scientific community copes with the burden it poses. We used mathematical modeling to estimate the overall quantitative annual demand for peer review and the supply in biomedical research. The modeling was informed by empirical data from various sources in the biomedical domain, including all articles indexed at MEDLINE. We found that for 2015, across a range of scenarios, the supply exceeded by 15% to 249% the demand for reviewers and reviews. However, 20% of the researchers performed 69% to 94% of the reviews. Among researchers actually contributing to peer review, 70% dedicated 1% or less of their research work-time to peer review while 5% dedicated 13% or more of it. An estimated 63.4 million hours were devoted to peer review in 2015, among which 18.9 million hours were provided by the top 5% contributing reviewers. Our results support that the system is sustainable in terms of volume but emphasizes a considerable imbalance in the distribution of the peer-review effort across the scientific community. Finally, various individual interactions between authors, editors and reviewers may reduce to some extent the number of reviewers who are available to editors at any point.

Fig 1. Input distributions and results derived from MEDLINE for peer review in the biomedical domain.

(A) Amount of annual publications indexed by MEDLINE and the demand for reviews they generate; (B) Peer-review effort for 2015 provided by Publons. The inset shows the distribution for more than 20 reviews completed per year. Data refer to all scientific domains; (C) Number of authors who published during a given year. Data are from analyzing all annual publications indexed by MEDLINE; (D) Distribution of time spent per review. Data are from Mulligan et al. (2011) and refers to the medical domain.

Fig 2. Comparison between supply and demand for reviewers and reviews.

(A) Supply and demand for reviewers for all author scenarios. (B) Supply and demand for reviews for all author scenarios.

Fig 3. Imbalance in the peer-review effort in terms of workload and work-time.

(A) Percentage of authors who complete a certain proportion of the peer-review workload for 2015. (B) Authors’ annual percentage of work-time devoted to peer review.