How blind is blind review?

Abstract

Background: No representative surveys of scientific opinion about blind review have been published, and there is very little information on the success of the blinding process. The American Journal of Public Health has practiced blind review since 1977.

Methods: In 1989 to 1990 312 of its reviewers were asked to identify author and institution in the manuscript they reviewed, to provide clues to such identification, to express their opinion concerning blind review, and to offer reasons for their opinion.

Results: Reviewers claimed to be able to identify author and/or institution in 47% of the 614 chances offered; identification was incorrect 16% of the time, overall identification correct 39% of the time. Self-referencing was the clue to identification in 62%, personal knowledge in 38% of the cases. If only personal knowledge cases are considered, blinding was successful 83% of the time. Blinding was favored by 75% of the reviewers with most asserting it eliminated bias. Reasons given for opposing blind review included the following: blinding not possible, identification will not influence judgment, and its obverse, identification assists judgment.

Conclusions: For the American Journal of Public Health blinding is usually, but not always, successful; and the majority of its reviewers favor current policy. Until more definitive data are in, reviewer preference, which differs from journal to journal, seems the most legitimate guide to journal policy on blind review.