Big Science vs. Little Science: How Scientific Impact Scales with Funding

Abstract

is it more effective to give large grants to a few elite researchers, or small grants to many researchers? Large grants would be more effective only if scientific impact increases as an accelerating function of grant size. Here, we examine the scientific impact of individual university-based researchers in three disciplines funded by the Natural Sciences and Engineering Research Council of Canada (NSERC). We considered four indices of scientific impact: numbers of articles published, numbers of citations to those articles, the most cited article, and the number of highly cited articles, each measured over a four-year period. We related these to the amount of NSERC funding received. Impact is positively, but only weakly, related to funding. Researchers who received additional funds from a second federal granting council, the Canadian Institutes for Health Research, were not more productive than those who received only NSERC funding. Impact was generally a decelerating function of funding. Impact per dollar was therefore lower for large grant-holders. This is inconsistent with the hypothesis that larger grants lead to larger discoveries. Further, the impact of researchers who received increases in funding did not predictably increase. We conclude that scientific impact (as reflected by publications) is only weakly limited by funding. We suggest that funding strategies that target diversity, rather than "excellence", are likely to prove to be more productive.

Figure 1. Four measures of the scientific impact of individual researchers from 2003 to 2007, expressed as functions of the logarithm of each researcher's NSERC Discovery grant (i.e. operating grant received from the Natural Sciences and Engineering Research Council of Canada) (left column), or the NSERC grant plus the researcher's grant from the Canadian Institutes for Health Research (CIHR), if any, in 2002 to 2006.

The measures of scientific impact are: the total numbers of papers published, numbers of citations to those publications by 2012, the number of citations received by the most highly cited paper, and the number of very highly cited papers. The solid lines represent LOWESS (model-free) fits to the data. Dashed lines show linear regression fits to the log-transformed data. Dotted lines show a slope of 1.0. Symbols distinguish researchers who held only an NSERC grant, versus those who also held a grant from CIHR, CFI (the Canadian Foundation for Innovation) and/or the Fonds Québécois de Recherche – Nature et Technologies (FQRNT). Results are shown for scientists funded by the NSERC grant selection committee in Integrative Animal Biology. In all cases, individual impact increases with funding with a slope ≤1.0. Thus, impact is a decelerating function of grant size. Researchers who held grants other than NSERC are not significantly more productive than those who did not.

Figure 2. Four measures of the scientific impact of individual researchers from 2003 to 2007, expressed as functions of the logarithm of each researcher's NSERC Discovery grant.

Details are the same as in Figure 1, except that results shown here are for scientists funded by the NSERC grant selection committee in Organic and Inorganic Chemistry. Again, individual impact increases with funding with a slope ≤1.0, and researchers who held grants other than NSERC are not significantly more productive than those who did not.

Figure 3. Four measures of the scientific impact of individual researchers from 2003 to 2007, expressed as functions of the logarithm of each researcher's NSERC Discovery grant.

Details are the same as in Figures 1 and 2, except that results shown here are for scientists funded by the NSERC grant selection committee in Ecology and Evolution. Once again, individual impact increases with funding with a slope ≤1.0, and researchers who held grants other than NSERC are not significantly more productive than those who did not.

Figure 4. For individual researchers in three disciplines, change in the number of papers published in 2007–2011, in comparison to 2003–2006, expressed as a function of the change in grant funding from the Natural Sciences and Engineering Research Council of Canada from 2006–2010, in comparison to 2002–2005.

The solid lines represent LOWESS fits to the data, while the dotted lines represent linear regressions. In all cases, there was no significant relationship.

Figure 5. Two measures of scientific productivity measured in 2003–2006 and 2007–2010 for researchers in Integrative Animal Biology.

Very similar results were observed for the two other disciplines studied here.