Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Nov 28;11(11):MR000056.
doi: 10.1002/14651858.MR000056.pub2.

Reviewer training for improving grant and journal peer review

Jan-Ole Hesselberg  1   2 , Therese K Dalsbø  3 , Hilde Stromme  4 , Ida Svege  2   5 , Atle Fretheim  5   6
Affiliations

Reviewer training for improving grant and journal peer review

Jan-Ole Hesselberg et al. Cochrane Database Syst Rev. .

Abstract

Background: Funders and scientific journals use peer review to decide which projects to fund or articles to publish. Reviewer training is an intervention to improve the quality of peer review. However, studies on the effects of such training yield inconsistent results, and there are no up-to-date systematic reviews addressing this question.

Objectives: To evaluate the effect of peer reviewer training on the quality of grant and journal peer review.

Search methods: We used standard, extensive Cochrane search methods. The latest search date was 27 April 2022.

Selection criteria: We included randomized controlled trials (RCTs; including cluster-RCTs) that evaluated peer review with training interventions versus usual processes, no training interventions, or other interventions to improve the quality of peer review.

Data collection and analysis: We used standard Cochrane methods. Our primary outcomes were 1. completeness of reporting and 2. peer review detection of errors. Our secondary outcomes were 1. bibliometric scores, 2. stakeholders' assessment of peer review quality, 3. inter-reviewer agreement, 4. process-centred outcomes, 5. peer reviewer satisfaction, and 6. completion rate and speed of funded projects. We used the first version of the Cochrane risk of bias tool to assess the risk of bias, and we used GRADE to assess the certainty of evidence.

Main results: We included 10 RCTs with a total of 1213 units of analysis. The unit of analysis was the individual reviewer in seven studies (722 reviewers in total), and the reviewed manuscript in three studies (491 manuscripts in total). In eight RCTs, participants were journal peer reviewers. In two studies, the participants were grant peer reviewers. The training interventions can be broadly divided into dialogue-based interventions (interactive workshop, face-to-face training, mentoring) and one-way communication (written information, video course, checklist, written feedback). Most studies were small. We found moderate-certainty evidence that emails reminding peer reviewers to check items of reporting checklists, compared with standard journal practice, have little or no effect on the completeness of reporting, measured as the proportion of items (from 0.00 to 1.00) that were adequately reported (mean difference (MD) 0.02, 95% confidence interval (CI) -0.02 to 0.06; 2 RCTs, 421 manuscripts). There was low-certainty evidence that reviewer training, compared with standard journal practice, slightly improves peer reviewer ability to detect errors (MD 0.55, 95% CI 0.20 to 0.90; 1 RCT, 418 reviewers). We found low-certainty evidence that reviewer training, compared with standard journal practice, has little or no effect on stakeholders' assessment of review quality in journal peer review (standardized mean difference (SMD) 0.13 standard deviations (SDs), 95% CI -0.07 to 0.33; 1 RCT, 418 reviewers), or change in stakeholders' assessment of review quality in journal peer review (SMD -0.15 SDs, 95% CI -0.39 to 0.10; 5 RCTs, 258 reviewers). We found very low-certainty evidence that a video course, compared with no video course, has little or no effect on inter-reviewer agreement in grant peer review (MD 0.14 points, 95% CI -0.07 to 0.35; 1 RCT, 75 reviewers). There was low-certainty evidence that structured individual feedback on scoring, compared with general information on scoring, has little or no effect on the change in inter-reviewer agreement in grant peer review (MD 0.18 points, 95% CI -0.14 to 0.50; 1 RCT, 41 reviewers, low-certainty evidence).

Authors' conclusions: Evidence from 10 RCTs suggests that training peer reviewers may lead to little or no improvement in the quality of peer review. There is a need for studies with more participants and a broader spectrum of valid and reliable outcome measures. Studies evaluating stakeholders' assessments of the quality of peer review should ensure that these instruments have sufficient levels of validity and reliability.

PubMed Disclaimer

Conflict of interest statement

JOH: together with IS, JOH is an author of one of the studies included in this review (Hesselberg 2021). For this study, AF and TD performed screening, data extraction, and risk of bias assessment. JOH is the chief programme officer at The Norwegian Foundation Dam (Stiftelsen Dam), a research funder that also funded the PhD which this systematic review is a part of. IS: together with JOH, is an author of one of the studies included in this review (Hesselberg 2021). For this study, AF and TD performed screening, data extraction, and risk of bias assessment. IS is the head of programme development at The Norwegian Foundation Dam (Stiftelsen Dam). TD: none known. HS: none known. AF: none known.

Figures

1
1
2
2
'Risk of bias' summary: review authors' judgements about each risk of bias item for each included study.
3
3
'Risk of bias; graph: review authors' judgements about each risk of bias item presented as percentages across all included trials.
1.1
1.1. Analysis
Comparison 1: Training versus no training or standard journal practice, Outcome 1: Completeness of reporting (proportion of items adequately reported, from 0.00 to 1.00)
1.2
1.2. Analysis
Comparison 1: Training versus no training or standard journal practice, Outcome 2: Peer reviewer detection of errors (number of errors identified, from 0 to 9)
1.3
1.3. Analysis
Comparison 1: Training versus no training or standard journal practice, Outcome 3: Stakeholders' assessment of review quality
1.4
1.4. Analysis
Comparison 1: Training versus no training or standard journal practice, Outcome 4: Inter‐reviewer agreement
See this image and copyright information in PMC

Update of

  • doi: 10.1002/14651858.MR000056

Similar articles

See all similar articles

Cited by

References

References to studies included in this review

Callaham 2002a Study 1 {published data only}
    1. Callaham ML, Knopp RK, Gallagher EJ. Effect of written feedback by editors on quality of reviews: two randomized trials. Journal of the American Medical Association 2002;287(21):2781-3. [DOI: 10.1001/jama.287.21.2781] - DOI - PubMed
Callaham 2002a Study 2 {published data only}
    1. Callaham ML, Knopp RK, Gallagher EJ. Effect of written feedback by editors on quality of reviews: two randomized trials. Journal of the American Medical Association 2002;287(21):2781-3. [DOI: 10.1001/jama.287.21.2781] - DOI - PubMed
Callaham 2002b {published data only}
    1. Callaham ML, Schriger DL. Effect of structured workshop training on subsequent performance of journal peer reviewers. Annals of Emergency Medicine 2002;40(3):323-8. - PubMed
Cobo 2007 {published data only}
    1. Cobo E, Selva-O'Callagham A, Ribera JM, Cardellach F, Dominguez R, Vilardell M. Statistical reviewers improve reporting in biomedical articles: a randomized trial. PLOS ONE 2007;2(3):e332. [DOI: 10.1371/journal.pone.0000332] - DOI - PMC - PubMed
Hesselberg 2021 {published data only}
    1. Hesselberg J-O, Fostervold KI, Ulleberg P, Svege I. Individual versus general structured feedback to improve agreement in grant peer review: a randomized controlled trial. Research Integrity and Peer Review 2021;6:12. [DOI: 10.21203/rs.3.rs-461626/v1] - DOI - PMC - PubMed
Houry 2012 {published data only}
    1. Houry D, Green S, Callaham M. Does mentoring new peer reviewers improve review quality? A randomized trial. BMC Medical Education 2012;12:83. [DOI: ] - PMC - PubMed
Sattler 2015 {published data only}
    1. Sattler DN, McKnight PE, Naney L, Mathis R. Grant peer review: improving inter-rater reliability with training. PLOS ONE 2015;10(6):e0130450. [DOI: 10.1371/journal.pone.0130450] - DOI - PMC - PubMed
Schroter 2004 {published data only}
    1. Schroter S, Black N, Evans S, Carpenter J, Godlee F, Smith R. Effects of training on quality of peer review: randomised controlled trial. BMJ 2004;328(7441):673. [DOI: 10.1136/bmj.38023.700775.AE] - DOI - PMC - PubMed
Speich 2023 Study 1 {unpublished data only}
    1. Speich B, Mann E, Schönenberger CM, Mellor K, Griessbach AN, Dhiman P, et al. Reminding peer reviewers of the most important reporting guideline items to improve completeness in published articles: primary results of two randomized controlled trials. Personal communication with author 2023. - PMC - PubMed
    1. Speich B. Impact of a short form of the SPIRIT checklist for peer reviewers to improve the reporting of protocols for randomised controlled trials published in biomedical journals: a randomised controlled trial. Open Science Framework 2020. [DOI: 10.17605/OSF.IO/Z2HM9] - DOI - PMC - PubMed
Speich 2023 Study 2 {published data only}
    1. Speich B, Mann E, Schönenberger CM, Mellor K, Griessbach AN, Dhiman P, et al. Reminding peer reviewers of the most important reporting guideline items to improve completeness in published articles: primary results of two randomized controlled trials. Personal communication with author 2023. - PMC - PubMed
    1. Speich B, Schroter S, Briel M, Moher D, Puebla I, Clark A, et al. Impact of a short version of the CONSORT checklist for peer reviewers to improve the reporting of randomised controlled trials published in biomedical journals: study protocol for a randomised controlled trial. BMJ Open 2020;10(3):e035114. [DOI: 10.1136/bmjopen-2019-035114] - DOI - PMC - PubMed

References to studies excluded from this review

Blanco 2020 {published data only}
    1. Blanco D, Schroter S, Aldcroft A, Moher D, Boutron I, Kirkham JJ, et al. Effect of an editorial intervention to improve the completeness of reporting of randomised trials: a randomised controlled trial. BMJ Open 2020;10(5):e036799. [DOI: 10.1136/bmjopen-2020-036799] - DOI - PMC - PubMed
Chauvin 2017 {published data only}
    1. Chauvin A, Moher D, Altman D, Schriger DL, Alam S, Hopewell S, et al. A protocol of a cross-sectional study evaluating an online tool for early career peer reviewers assessing reports of randomised controlled trials. BMJ Open 2017;7(9):1-10. [DOI: 10.1136/bmjopen-2017-017462] - DOI - PMC - PubMed
Cobo 2011 {published data only}
    1. Cobo E, Cortes J, Ribera JM, Cardellach F, Selva-O'Callaghan A, Kostov B, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ 2011;343:d6783. [DOI: 10.1136/bmj.d6783] - DOI - PMC - PubMed
Crowe 2011 {published data only}
    1. Crowe M, Sheppard L, Campbell A. Comparison of the effects of using the Crowe Critical Appraisal Tool versus informal appraisal in assessing health research: a randomised trial. International Journal of Evidence-Based Healthcare 2011;9(4):444-9. [DOI: 10.1111/j.1744-1609.2011.00237.x] - DOI - PubMed
DiDomenico 2017 {published data only}
    1. DiDomenico RJ, Baker WL, Haines AT. Improving peer review: what reviewers can do. American Journal of Health-System Pharmacy 2017;74(24):2080-4. [DOI: 10.2146/ajhp170190] - DOI - PubMed
Gardner 1986 {published data only}
    1. Gardner MJ, Machin D, Campbell MJ. Use of check lists in assessing the statistical content of medical studies. British Medical Journal (Clinical Research Edition) 1986;292(6523):810-2. [DOI: 10.1136/bmj.292.6523.810] - DOI - PMC - PubMed
Ghannad 2021 {published data only}
    1. Ghannad M, Yang B, Leeflang M, Aldcroft A, Bossuyt PM, Schroter S, et al. A randomized trial of an editorial intervention to reduce spin in the abstract's conclusion of manuscripts showed no significant effect. Journal of Clinical Epidemiology 2021;130:69-77. [DOI: 10.1016/j.jclinepi.2020.10.014] - DOI - PubMed
Green 1989 {published data only}
    1. Green JG, Calhoun F, Nierzwicki L, Brackett J, Meier P. Rating intervals: an experiment in peer review. FASEB Journal 1989;3(8):1987-92. [DOI: 10.1096/fasebj.3.8.2721858] - DOI - PubMed
Jones 2019 {published data only}
    1. Jones CW, Adams A, Weaver MA, Schroter S, Misemer BS, Schriger D, et al. Peer reviewed evaluation of registered end-points of randomised trials (the PRE-REPORT study): protocol for a stepped-wedge, cluster-randomised trial. BMJ Open 2019;12:e066624. - PMC - PubMed
MacAuley 1998 {published data only}
    1. MacAuley D, McCrum E, Brown C. Randomised controlled trial of the READER method of critical appraisal in general practice. BMJ 1998;316(7138):1134-7. [DOI: 10.1136/bmj.316.7138.1134] - DOI - PMC - PubMed
NCT03751878 {published data only}
    1. NCT03751878. Evaluating the impact of assessing during peer review the CONSORT checklist submitted by authors. clinicaltrials.gov/study/NCT03751878?tab=history (first received 21 November 2018).
Pitman 2019 {published data only}
    1. Pitman A, Underwood R, Hamilton A, Tyrer P, Yang M. Enhanced peer-review for optimising publication of biomedical papers submitted from low- and middle-income countries: feasibility study for a randomised controlled trial. BJPsych Open 2019;5(2):e20. [DOI: 10.1192/bjo.2018.89] - DOI - PMC - PubMed
Strowd 2014 {published data only}
    1. Strowd R, Wong V, Aragon-Garcia R, Moon Y, Ford B, Haut S, et al. Use of mentored peer review of standardized manuscripts as an educational tool for neurology residents (P1.335). Neurology 2014;82(10 Supplement):P1.335.
Wong 2017 {published data only}
    1. Wong VS, Strowd RE, Aragon-Garcia R, Moon YO, Ford B, Haut SR, et al. Mentored peer review of standardized manuscripts as a teaching tool for residents: a pilot randomized controlled multi-center study. Research Integrity & Peer Review 2017;2:6. [DOI: 10.1186/s41073-017-0032-0] - DOI - PMC - PubMed

Additional references

Abdoul 2012
    1. Abdoul H, Perrey C, Amiel P, Tubach F, Gottot S, Durand-Zaleski I, et al. Peer review of grant applications: criteria used and qualitative study of reviewer practices. PLOS One 2012;7(9):e46054. - PMC - PubMed
Bhargava 2014
    1. Bhargava S, Fisman R. Contrast effects in sequential decisions: evidence from speed dating. Review of Economics and Statistics 2014;96(3):444-57.
Bol 2018
    1. Bol T, Vaan M, de Rijt A. The Matthew effect in science funding. Proceedings of the National Academy of Sciences of the United States of America 2018;115(19):4887-90. - PMC - PubMed
Bornmann 2006
    1. Bornmann L, Daniel H-D. Potential sources of bias in research fellowship assessments: effects of university prestige and field of study. Research Evaluation 2006;15(3):209-19.
Bornmann 2011
    1. Bornmann L. Scientific peer review. Annual Review of Information Science and Technology 2011;45(1):197-245.
Bruce 2016
    1. Bruce R, Chauvin A, Trinquart L, Ravaud P, Boutron I. Impact of interventions to improve the quality of peer review of biomedical journals: a systematic review and meta-analysis. BMC Medicine 2016;14(1):85. - PMC - PubMed
Callaham 1998
    1. Callaham ML, Baxt WG, Waeckerle JF, Wears RL. Reliability of editors’ subjective quality ratings of peer reviews of manuscripts. Journal of the American Medical Association 1998;280(3):229-31. [DOI: 10.1001/jama.280.3.229] - DOI - PubMed
Chan 2013
    1. Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Annals of Internal Medicine 2013;158(3):200-7. - PMC - PubMed
Chauvin 2015
    1. Chauvin A, Ravaud P, Baron G, Barnes C, Boutron I. The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. BMC Medicine 2015;13:158. - PMC - PubMed
Covidence 2021 [Computer program]
    1. Covidence. Version accessed 29 May 2019. Melbourne, Australia: Veritas Health Innovation, 2021. Available at covidence.org.
Danziger 2011
    1. Danziger S, Levav J, Avnaim-Pesso L. Extraneous factors in judicial decisions. Proceedings of the National Academy of Sciences of the United States of America 2011;108(17):6889-92. - PMC - PubMed
Deeks 2017
    1. Deeks JJ, Higgins JP, Altman DG, editor(s) on behalf of the Cochrane Statistical Methods Group. Chapter 9: Analysing data and undertaking meta‐analyses. In: Higgins JP, Churchill R, Chandler J, Cumpston MS, editor(s), Cochrane Handbook for Systematic Reviews of Interventions Version 5.2.0 (updated June 2017). Cochrane, 2017. Available from www.training.cochrane.org/handbook.
Demicheli 2007
    1. Demicheli V, Di Pietrantonj C. Peer review for improving the quality of grant applications. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No: MR000003. [DOI: 10.1002/14651858.MR000003.pub2] - DOI - PMC - PubMed
Egger 1997
    1. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ (Clinical Research Ed.) 1997;315(7109):629-34. - PMC - PubMed
Elsevier 2023
    1. Elsevier. Role of a reviewer. www.elsevier.com/reviewers/role (accessed 11 Mar 2023).
European Commission 2018
    1. European Commission. HORIZON 2020 in full swing. Three years on. Key facts and figures 2014-2016. European Commission, 2018. Available from: www.kowi.de/Portaldata/2/Resources/horizon2020/H2020-2014-2016-Key-Facts....
European Commission 2021
    1. Directorate-General for Research and Innovation. Horizon Europe, budget: Horizon Europe - the most ambitious EU research & innovation programme ever. Publications Office of the European Union 2021. [DOI: 10.2777/202859] - DOI
Gallo 2018
    1. Gallo S, Thompson L, Schmaling K, Glisson S. Risk evaluation in peer review of grant applications. Environment Systems and Decisions 2018;38(2):216-29.
Goodman 1994
    1. Goodman SN. Manuscript quality before and after peer review and editing at Annals of Internal Medicine. Annals of Internal Medicine 1994;121(1):11-21. [DOI: 10.7326/0003-4819-121-1-199407010-00003] - DOI - PubMed
Guthrie 2017
    1. Guthrie S, Ghiga I, Wooding S. What do we know about grant peer review in the health sciences? F1000Research 2017;6:1335. - PMC - PubMed
Guyatt 2008
    1. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ (Clinical Research Ed.) 2008;336(7650):924-6. - PMC - PubMed
Harbord 2006
    1. Harbord RM, Egger M, Sterne JA. A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Statistics in Medicine 2006;25(20):3443-57. - PubMed
Higgins 2003
    1. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ (Clinical Research Ed.) 2003;327(7414):557-60. - PMC - PubMed
Higgins 2011
    1. Higgins JP, Green S, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (updated March 2011). The Cochrane Collaboration, 2011. Available from handbook.cochrane.org.
Higgins 2017
    1. Higgins JP, Altman DG, Sterne JA. Chapter 8: Assessing risk of bias in included studies. In: Higgins JP, Churchill R, Chandler J, Cumpston MS, editor(s). Cochrane Handbook for Systematic Reviews of Interventions Version 5.2.0 (updated June 2017). The Cochrane Collaboration, 2017. Available from www.training.cochrane.org/handbook.
Hoffmann 2014
    1. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ (Clinical Research Ed.) 2014;348:g1687. - PubMed
Horsley 2011
    1. Horsley T, Dingwall O, Sampson M. Checking reference lists to find additional studies for systematic reviews. Cochrane Database of Systematic Reviews 2011, Issue 8. Art. No: MR000026. [DOI: 10.1002/14651858.MR000026.pub2] - DOI - PMC - PubMed
Huber 2022
    1. Huber J, Sabiou A, Kerschbamer, Smith VL. Nobel and novice: author prominence affects peer review. Proceedings of the National Academy of Sciences of the United States of America 2022;119(41):e2205779119. [DOI: 10.1073/pnas.2205779119] - DOI - PMC - PubMed
Jefferson 2007
    1. Jefferson T, Rudin M, Brodney Folse S, Davidoff F. Editorial peer review for improving the quality of reports of biomedical studies. Cochrane Database of Systematic Reviews 2007, Issue 2. Art. No: MR000016. [DOI: 10.1002/14651858.MR000016.pub3] - DOI - PMC - PubMed
Lane 2020
    1. Lane JN, Teplitskiy M, Gray G, Ranu H, Menietti M, Guinan E, et al. Conservatism Gets Funded? A Field Experiment on the Role of Negative Information in Novel Project Evaluation. Harvard Business School Technology & Operations Mgt. Unit Working Paper No. 21-007, 2020. Available from: papers.ssrn.com/sol3/papers.cfm?abstract_id=3656495. [WEBSITE: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3656495]
Langfeldt 2006
    1. Langfeldt L. The policy challenges of peer review: managing bias, conflict of interests and interdisciplinary assessments. Research Evaluation 2006;15(1):31-41.
Lee 2013
    1. Lee CJ, Sugimoto CR, Zhang G, Cronin B. Bias in peer review. Journal of the American Society for Information Science and Technology 2013;64(1):2-17.
Mantel 1959
    1. Mantel N, Haenszel W. Statistical aspects of the analysis of data from retrospective studies of disease. JNCI: Journal of the National Cancer Institute 1959;22(4):719-48. - PubMed
Mulligan 2013
    1. Mulligan A, Hall L, Raphael E. Peer review in a changing world: an international study measuring the attitudes of researchers. Journal of the American Society for Information Science and Technology 2013;64(1):132-61.
National Institutes of Health 2022
    1. National Institutes of Health. Budget. www.nih.gov/about-nih/what-we-do/budget (accessed 11 March 2023). [WEBSITE: https://www.nih.gov/about-nih/what-we-do/budget]
National Library of Medicine 2022
    1. National Library of Medicine. MEDLINE PubMed Production Statistics. National Library of Medicine 2022. [WEBSITE: https://www.nlm.nih.gov/bsd/medline_pubmed_production_stats.html]
Olbrecht 2010
    1. Olbrecht M, Bornmann L. Panel peer review of grant applications: what do we know from research in social psychology on judgment and decision-making in groups? Research Evaluation 2010;19(4):293-304.
Page 2021
    1. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 2021;371:n71. [DOI: 10.1136/bmj.n71] - DOI - PMC - PubMed
Recio‐Saucedo 2022
    1. Recio-Saucedo A, Crane K, Meadmore K, Fackrell K, Church H, Fraser S, et al. What works for peer review and decision-making in research funding: a realist synthesis. Research Integrity and Peer Review 2022;7(2):1-28. [DOI: 10.1186/s41073-022-00120-2] - DOI - PMC - PubMed
RevMan Web 2022 [Computer program]
    1. Review Manager Web (RevMan Web). Version 4.12.0. The Cochrane Collaboration, 2022. Available at revman.cochrane.org.
Roumbanis 2017
    1. Roumbanis L. Academic judgments under uncertainty: A study of collective anchoring effects in Swedish Research Council panel groups. Social Studies of Science 2017;47(1):95-116. [DOI: 10.1177/0306312716659789] - DOI - PubMed
Sattler 2015
    1. Sattler DN, McKnight PE, Naney L, Mathis R. Grant peer review: improving inter-rater reliability with training. PLOS One 2015;10(6):e0130450. - PMC - PubMed
Schulz 2010
    1. Schulz KF, Altman DG, Moher D, the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMC Medicine 2010;8:18. [DOI: 10.1186/1741-7015-8-18] - DOI - PMC - PubMed
Sense About Science 2009
    1. Sense About Science. Peer Review Survey 2009: Full Report. senseaboutscience.org/activities/peer-review-survey-2009/ (accessed 30 July 2020).
Superchi 2019
    1. Superchi C, González JA, Solà I, Cobo E, Hren D, Boutron I. Tools used to assess the quality of peer review reports: a methodological systematic review. BMC Medical Research Methodology 2019;19(1):48. [DOI: 10.1186/s12874-019-0688-x] - DOI - PMC - PubMed
Teplitskiy 2019
    1. Teplitskiy M, Gray G, Guinan E, Ranu H, Menietti M, Lakhani KR. Do Experts Listen to Other Experts? Field Experimental Evidence from Scientific Peer Review. Harvard Business School, 2019.. Avaialble from www.hbs.edu/ris/Publication%20Files/19-107_06115731-d0ae-4a11-ab1d-ecaec....
van Rooyen 1999
    1. Rooyen S, Black N, Godlee F. Development of the review quality instrument (RQI) for assessing peer reviews of manuscripts. Journal of Clinical Epidemiology 1999;52(7):625-9. - PubMed
Walker 2015
    1. Walker R, Barros B, Conejo R, Neumann K, Telefont M. Personal attributes of authors and reviewers, social bias and the outcomes of peer review: a case study. F1000Research 2015;4:21. - PMC - PubMed
Wang 2015
    1. Wang Q, Sandström U. Defining the role of cognitive distance in the peer review process with an explorative study of a grant scheme in infection biology. Research Evaluation 2015;24(3):271-81.
Ware 2015
    1. Ware M, Mabe M. The STM Report - an overview of scientific and scholarly journal publishing. Available from www.stm-assoc.org/2015_02_20_STM_Report_2015.pdf 2015.
Warne 2016
    1. Warne V. Rewarding reviewers – sense or sensibility? A Wiley study explained. Learned Publishing 2016;29(1):41-50.
Young 2011
    1. Young T, Hopewell S. Methods for obtaining unpublished data. Cochrane Database of Systematic Reviews 2011, Issue 11. Art. No: MR000027. [DOI: 10.1002/14651858.MR000027.pub2] - DOI - PMC - PubMed

References to other published versions of this review

Hesselberg 2020
    1. Hesselberg J-O, Dalsbø TK, Stromme H, Svege I, Fretheim A. Reviewer training for improving grant and journal peer review. Cochrane Database of Systematic Reviews 2020, Issue 11. Art. No: MR000056. [DOI: 10.1002/14651858.MR000056] - DOI - PMC - PubMed

Publication types

LinkOut - more resources