Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 May 16;9(1):e129.
doi: 10.1017/cts.2025.76. eCollection 2025.

Measuring the aggregated impact of research: Establishing criteria for coding Translational Science Benefits Model data

Nicole Miovsky  1 , Amanda Woodworth  1 , Stephanie Andersen  2 , Rosalina Das  3 , Julie Heidbreder  4 , Rechelle Paranal  5 , Clara M Pelfrey  6 , Jessica Sperling  7   8 , Beth Tigges  9 , Boris B Volkov  10 , Margaret Schneider  1
Affiliations

Measuring the aggregated impact of research: Establishing criteria for coding Translational Science Benefits Model data

Nicole Miovsky et al. J Clin Transl Sci. .

Abstract

Introduction: A promising approach to assessing research impact draws on the Translational Science Benefits Model (TSBM), an evaluation model that tracks the applied benefits of research in four domains: Clinical and Medical; Community and Public Health; Economic; and Policy and Legislative. However, standardized methods to verify TSBM benefit data, to aid in aggregating impact data within quantitative summaries, do not currently exist.

Methods: A panel of 11 topic experts participated in a modified Delphi process for establishing content and face validity of a set of criteria for verifying qualitative TSBM data. Two survey rounds were completed by panelists, with a moderated discussion in between rounds to discuss criteria not reaching consensus. Criteria with panel consensus at or above 70% in the survey rounds were confirmed as validated.

Results: Criteria fell into 9 categories: Content Relevant, Project Related, Who, Reach, What, How, Novel, Documented Evidence, and When. The Delphi process yielded 197 total criteria across the 30 benefits characterized by the TSBM (range = 5-8 criteria per benefit).

Discussion: The results of this Delphi process lay the foundation for developing a TSBM coding tool for evaluating and quantifying TSBM data. Standardizing this process will enable data aggregation, group analysis, and the comparison of research impact across contexts.

Keywords: Delphi panel; Research impact evaluation; coding criteria; content validation; health research; societal impact.

PubMed Disclaimer

Conflict of interest statement

The author(s) declare none.

Figures

Figure 1.
Figure 1.
Flowchart of criteria across the Delphi panel rounds.
Figure 2.
Figure 2.
Validated criteria from the Delphi panel.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Tsey K, Lawson K, Kinchin I, et al. Evaluating research impact: the development of a research for impact tool. Front Public Health. 2016;4:160. doi: 10.3389/fpubh.2016.00160. - DOI - PMC - PubMed
    1. Molas J, Tang P, Morrow S. Assessing the non-academic impact of grant-funded socio-economic research: Results from a pilot study. Res Evaluat. 2000:9:171–182. doi: 10.3152/147154400781777269. - DOI
    1. Milat AJ, Bauman AE, Redman S. A narrative review of research impact assessment models and methods. Health Res Policy Sys. 2015;13(1):18. doi: 10.1186/s12961-015-0003-1. - DOI - PMC - PubMed
    1. Reed MS, Ferré M, Martin-Ortega J, et al. Evaluating impact from research: a methodological framework. Res Policy. 2021;50(4):104147. doi: 10.1016/j.respol.2020.104147. - DOI
    1. Bornmann L. Measuring impact in research evaluations: A thorough discussion of methods for, effects of and problems with impact measurements. High Educ. 2017;73(5):775–787. doi: 10.1007/s10734-016-9995-x. - DOI

LinkOut - more resources