Variation in journal peer review systems. Possible causes and consequences

Abstract

To investigate disciplinary differences in how scientific journals evaluate submissions, I collected data from the Astrophysical Journal, Physiological Zoology, and the American Sociological Review. Referees' evaluations of submissions to these journals differed strikingly: nearly half of the referee reports for American Sociological Review recommended outright rejection, while the corresponding proportions for the other two journals were about one fourth and one tenth. Final dispositions show even greater variation, with Astrophysical Journal accepting 91% of submissions, Physiological Zoology 59%, and American Sociological Review 13%. The journals also differed substantially in the average number of revisions of eventually accepted papers and the average time lags between submission and final editorial decisions. Such differences conform to claims that disciplinary differences in consensus on research priorities and procedures contribute to variation in typical journal peer review systems. Review systems, in turn, influence authors' experiences in the peer review process.