How Criterion Scores Predict the Overall Impact Score and Funding Outcomes for National Institutes of Health Peer-Reviewed Applications

Abstract

Understanding the factors associated with successful funding outcomes of research project grant (R01) applications is critical for the biomedical research community. R01 applications are evaluated through the National Institutes of Health (NIH) peer review system, where peer reviewers are asked to evaluate and assign scores to five research criteria when assessing an application's scientific and technical merit. This study examined the relationship of the five research criterion scores to the Overall Impact score and the likelihood of being funded for over 123,700 competing R01 applications for fiscal years 2010 through 2013. The relationships of other application and applicant characteristics, including demographics, to scoring and funding outcomes were studied as well. The analyses showed that the Approach and, to a lesser extent, the Significance criterion scores were the main predictors of an R01 application's Overall Impact score and its likelihood of being funded. Applicants might consider these findings when submitting future R01 applications to NIH.

Fig 1. Box Plot Distributions of Criterion and Overall Impact Scores for R01 Applications, FY 2010–2013.

Fig 1 shows the box plot distributions of the five research criterion scores (scale: 1–9) and the Overall Impact score (scale: 10–90). Box plot whiskers extend to the most extreme data point which is no more than 1.5 times the interquartile range from the box. Each criterion score N = 123,707 applications; Overall Impact score N = 71,651 applications.

Fig 2. Box Plot Distributions of Overall Impact Scores for R01 Applications by IC, FY 2010–2013.

Fig 2 shows the box plot distributions of the Overall Impact score (scale: 10–90) by IC. Box plot whiskers extend to the most extreme data point which is no more than 1.5 times the interquartile range from the box. IC names have been masked. N = 71,651 applications (discussed applications only).

Fig 3. Distributions of Funding Rate for R01 Applications by IC, FY 2010–2013.

Fig 3 shows the distribution of the percentage of reviewed applications funded by each IC. IC names have been masked and have been labeled to agree with Fig 3, i.e., the IC labeled as “1” in Fig 2 is the same IC labeled as “1” in Fig 2. N = 123,707.