Bibliometric Approach to Evaluating the Impact of a Building Interdisciplinary Research Careers in Women's Health K12 Research Career Development Program

Abstract

Background: Mentored research career development programs are excellent training opportunities for junior faculty/early-stage investigators to transition into independent research careers. However, there is limited evidence that provides guidance on best practices for measuring the impact and reach of these programs, both for individual Scholars and the program as a whole. This article evaluates both the individual and overall impact of the Building Interdisciplinary Research Careers in Women's Health (BIRCWH) National Institutes of Health research career development award at the University of Minnesota. Materials and Methods: BIRCWH Scholars (n = 16) and a comparison group (n = 17) were evaluated on traditional metrics (e.g., publications, grant funding) in addition to bibliometrics (e.g., network growth, interdisciplinary collaborations, international reach, policy impact). Results: Traditional metric findings showed that BIRCWH Scholars had significantly more publications from pre- to post-BIRCWH experience than the comparison group and more grant funding. Bibliometric findings showed exponential network growth, interdisciplinary collaborations, international citations, and policy impact from pre- to post-BIRCWH Scholar experience. Conclusion: Findings from this evaluation have potential important implications. At the Scholar level, the results can be used to provide evidence of research impact in materials developed for merit review and promotion as well as in job and research grant applications. At the program level, the results can be used at the institutional level to gain broad administrative support and leverage additional funds for program activities and for evidence of program success for continuation funding from federal agencies.

Keywords: bibliometrics; interdisciplinary; research career development awards; sex differences; women's health.

FIG. 1.

Pre- and post-BIRCWH Network Growth Impact Example Depictions of one BIRCWH Scholar's collaboration network growth, including on the left (A) the Scholar's collaboration network before their appointment as a BIRCWH Scholar, and on the right (B) the Scholar's collaboration network after their appointment (e.g., February 2020). The circle size relates to the prevalence of collaborations (e.g., grants, publications) with the larger circle meaning a larger number of collaborations associated with that individual Scholar. The thickness of the network lines relates to the strength of the collaboration (e.g., number of coauthors). The colors represent different collaboration clusters. BIRCWH, Building Interdisciplinary Research Careers in Women's Health.

FIG. 2.

Pre- and post-BIRCWH International Impact Example Depictions of one BIRCWH Scholar's international impact, including on the top (A) the Scholar's cited publications on an international scale before their appointment as a BIRCWH Scholar, and on the bottom (B) the cited publications on an international scale after their appointment (e.g., February 2020). Each dot on the map represents citation(s).