Manuscript quality before and after peer review and editing at Annals of Internal Medicine

Abstract

Objective: To evaluate the effects of peer review and editing on manuscript quality.

Setting: Editorial offices of Annals of Internal Medicine.

Design: Masked before-after study. MANUSCRIPTS: 111 consecutive original research manuscripts accepted for publication at Annals between March 1992 and March 1993.

Measurements: We used a manuscript quality assessment tool of 34 items to evaluate the quality of the research report, not the quality of the research itself. Each item was scored on a 1 to 5 scale. Forty-four expert assessors unaware of the design or aims of the study evaluated the manuscripts, with different persons evaluating the two versions of each manuscript (before and after the editorial process).

Results: 33 of the 34 items changed in the direction of improvement, with the largest improvements seen in the discussion of study limitations, generalizations, use of confidence intervals, and the tone of conclusions. Overall, the percentage of items scored three or more increased by an absolute 7.3% (95% CI, 3.3% to 11.3%) from a baseline of 75%. The average item score improved by 0.23 points (CI, 0.07 to 0.39) from a baseline mean of 3.5. Manuscripts rated in the bottom 50% showed two- to threefold larger improvements than those in the top 50%, after correction for regression to the mean.

Conclusions: Peer review and editing improve the quality of medical research reporting, particularly in those areas that readers rely on most heavily to decide on the importance and generalizability of the findings.