Is it becoming harder to secure reviewers for peer review? A test with data from five ecology journals

Abstract

Background: There is concern in the academic publishing community that it is becoming more difficult to secure reviews for peer-reviewed manuscripts, but much of this concern stems from anecdotal and rhetorical evidence.

Methods: We examined the proportion of review requests that led to a completed review over a 6-year period (2009-2015) in a mid-tier biology journal (Molecular Ecology). We also re-analyzed previously published data from four other mid-tier ecology journals (Functional Ecology, Journal of Ecology, Journal of Animal Ecology, and Journal of Applied Ecology), looking at the same proportion over the period 2003 to 2010.

Results: The data from Molecular Ecology showed no significant decrease through time in the proportion of requests that led to a review (proportion in 2009 = 0.47 (95 % CI = 0.43 to 0.52), proportion in 2015 = 0.44 (95 % CI = 0.40 to 0.48)). This proportion did decrease for three of the other ecology journals (changes in proportions from 2003 to 2010 = -0.10, -0.18, and -0.09), while the proportion for the fourth (Functional Ecology) stayed roughly constant (change in proportion = -0.04).

Conclusions: Overall, our data suggest that reviewer agreement rates have probably declined slightly but not to the extent suggested by the anecdotal and rhetorical evidence.

Keywords: Academic journals; Peer review; Reviewers.

Fig. 1

Molecular Ecology data, plotted by year. a Distribution of the number of review invitations sent between 2009 and 2015 by year. The black lines are the medians, the boxes indicate the interquartile range, the whiskers extend to 1.5 times the interquartile range (IQR), and the points are outliers beyond 1.5*IQR. b Number of manuscripts with different numbers of completed reviews by year. c Mean proportion of completed reviews by year with the bottom panel showing the impact factor in grey crosses. The error bars indicate the 95 % confidence intervals

Fig. 2

Molecular Ecology data, plotted by editorial decision type. a Distribution of the number of review invitations sent for manuscripts with different editorial decisions. The black lines are the medians, the boxes indicate the interquartile range, the whiskers extend to 1.5 times the interquartile range (IQR), and the points are outliers beyond 1.5*IQR. b Number of manuscripts with different numbers of completed reviews by decision type. c Mean proportion of completed reviews by decision. The error bars indicate the 95 % confidence intervals

Fig. 3

Proportion of review requests that lead to a completed review for four ecology journals (a, b, c, d), 2003 to 2010 shown in solid circles. Data from [17]. Error bars indicate 95 % confidence intervals. Right-hand axis shows the impact factor for each journal (grey crosses)

Fig. 4

Comparison of proportion of review requests that were completed [16], filled circles, or agreed [5], open circles, for Functional Ecology between 2003 and 2014

Fig. 5

Left column: comparison of the number of reviewed submissions, and the number of unique reviewers invited for four ecology journals, 2003 to 2010. Data from Petchey et al. (2014b). Right column: comparison of the average number of review invitations per unique reviewer for four ecology journals, 2003 to 2010. Error bars indicate 95 % confidence intervals. Data from Petchey et al. (2014b)