Same review quality in open versus blinded peer review in "Ugeskrift for Læger"

Abstract

Introduction: Research into the peer review process has previously been conducted in English-language journals. This study deals with a Danish general medical journal with a relatively small pool of both reviewers and readers. The aim of the study was to compare the quality of reviews produced by identifiable and anonymous reviewers, and further to characterize authors' and reviewers' attitudes towards different peer review systems.

Material and methods: The study was conducted as a blinded, randomised controlled trial. Each manuscript was reviewed by an identifiable and an anonymous reviewer. Review quality was subsequently assessed by two blinded editors, using the validated Review Quality Instrument. Reviewers' and authors' attitudes towards different peer review systems were characterized using questionnaires.

Results: The study included 364 reviews. There was no statistically significant difference in quality between anonymous and identifiable reviewers' evaluations. 55% of the authors preferred the evaluation produced by the identifiable reviewer (p < 0.05). 26% of the identifiable reviewers found it unpleasant that authors knew their identities; 43% of the anonymous reviewers found it reassuring that authors did not know their identities. Regarding reviewers' preferences for different peer review systems, 38% preferred a double-blinded, 34% preferred a single-blinded and 28% preferred an open system. For authors, the corresponding proportions were 43%, 19% and 37%.

Conclusion: Implementing open peer review will not affect review quality, but lack of anonymity may cause reviewers, already limited in number, to decline when asked to review. Even though this would be a serious implication for a national journal like the Ugeskrift for Læger, the implementation of an open system should be discussed.

Funding: not relevant.

Trial registration: not relevant.