Does exchanging comments of Indian and non-Indian reviewers improve the quality of manuscript reviews?

Abstract

Background: The quality of peer reviewing in developing countries is thought to be poor. To examine whether this was so, we compared the performance of Indian and non-Indian reviewers who were sent original and review articles submitted to The National Medical Journal of India. We also tested whether informing reviewers that their comments would be exchanged improved the quality of their reviews.

Methods: In a prospective, randomized, blinded study, we sent 100 manuscripts to pairs of peer reviewers (Indian and non-Indian) of which 78 pairs of completed replies were available for analysis. Thirty-eight pairs of reviews were exchanged and 40 were not. The quality of the reviews was assessed by two editors who were unaware of the reviewers' nationality and whether they had been told that their reviews would be exchanged. The quality of the reviews was scored out of 100 (based on a predesigned evaluation proforma). We also measured the time taken to return a manuscript.

Results: Overall, non-Indian reviewers scored higher than Indians (mean scores non-Indians first, 56.7 v. 48.6, p < 0.001), especially those in the non-exchanged group (58.4 v. 47.3, p < 0.001) but not the exchanged group (54.8 v. 50.0, p < 0.06). Being informed that reviews would be exchanged did not affect the quality of reviews by non-Indians (54.8 exchanged v. 58.4 non-exchanged) or of reviews by Indians (50.0 exchanged v. 47.3 non-exchanged). The editors' assessment of the reviewers matched well (r = 0.59, p < 0.001). Non-Indians took the same amount of time as Indians to return their reviews, although the postage time was at least eight days longer.

Conclusions: We found that non-Indian peer reviewers were better than Indians and informing them that their views would be exchanged did not seem to affect the quality of their reviews. We suggest that Indian editors should also use non-Indian reviewers and start training programmes to improve the quality of peer reviews in India.