Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2018 May 23:3:4.
doi: 10.1186/s41073-018-0048-0. eCollection 2018.

Reviewing the review: a qualitative assessment of the peer review process in surgical journals

Catherine H Davis  1   2 , Barbara L Bass  2 , Kevin E Behrns  3 , Keith D Lillemoe  4 , O James Garden  5 , Mark S Roh  6 , Jeffrey E Lee  1 , Charles M Balch  1 , Thomas A Aloia  1
Affiliations

Reviewing the review: a qualitative assessment of the peer review process in surgical journals

Catherine H Davis et al. Res Integr Peer Rev. .

Abstract

Background: Despite rapid growth of the scientific literature, no consensus guidelines have emerged to define the optimal criteria for editors to grade submitted manuscripts. The purpose of this project was to assess the peer reviewer metrics currently used in the surgical literature to evaluate original manuscript submissions.

Methods: Manuscript grading forms for 14 of the highest circulation general surgery-related journals were evaluated for content, including the type and number of quantitative and qualitative questions asked of peer reviewers. Reviewer grading forms for the seven surgical journals with the higher impact factors were compared to the seven surgical journals with lower impact factors using Fisher's exact tests.

Results: Impact factors of the studied journals ranged from 1.73 to 8.57, with a median impact factor of 4.26 in the higher group and 2.81 in the lower group. The content of the grading forms was found to vary considerably. Relatively few journals asked reviewers to grade specific components of a manuscript. Higher impact factor journal manuscript grading forms more frequently addressed statistical analysis, ethical considerations, and conflict of interest. In contrast, lower impact factor journals more commonly requested reviewers to make qualitative assessments of novelty/originality, scientific validity, and scientific importance.

Conclusion: Substantial variation exists in the grading criteria used to evaluate original manuscripts submitted to the surgical literature for peer review, with differential emphasis placed on certain criteria correlated to journal impact factors.

Keywords: Journal reviewer; Manuscript review; Surgery journals; Surgical manuscripts; Surgical research.

PubMed Disclaimer

Conflict of interest statement

Not applicable.The authors declare that they have no competing interests.Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

References

    1. Manchikanti L, Kaye AD, Boswell MV, Hirsch JA. Medical journal peer review: process and bias. Pain Physician. 2015;18(1):E1–E14. - PubMed
    1. Burnham JC. The evolution of editorial peer review. JAMA. 1990;263(10):1323–1329. doi: 10.1001/jama.1990.03440100023003. - DOI - PubMed
    1. Patel J. Why training and specialization is needed for peer review: a case study of peer review for randomized controlled trials. BMC Med. 2014;12:128. doi: 10.1186/s12916-014-0128-z. - DOI - PMC - PubMed
    1. Stahel PF, Moore EE. Peer review for biomedical publications: we can improve the system. BMC Med. 2014;12:179. doi: 10.1186/s12916-014-0179-1. - DOI - PMC - PubMed
    1. McCulloch P, Feinberg J, Philippou Y, et al. Progress in clinical research in surgery and IDEAL. Lancet. 2018; - PubMed

LinkOut - more resources