Feasibility of common bibliometrics in evaluating translational science

Abstract

Introduction: A pilot study by 6 Clinical and Translational Science Awards (CTSAs) explored how bibliometrics can be used to assess research influence.

Methods: Evaluators from 6 institutions shared data on publications (4202 total) they supported, and conducted a combined analysis with state-of-the-art tools. This paper presents selected results based on the tools from 2 widely used vendors for bibliometrics: Thomson Reuters and Elsevier.

Results: Both vendors located a high percentage of publications within their proprietary databases (>90%) and provided similar but not equivalent bibliometrics for estimating productivity (number of publications) and influence (citation rates, percentage of papers in the top 10% of citations, observed citations relative to expected citations). A recently available bibliometric from the National Institutes of Health Office of Portfolio Analysis, examined after the initial analysis, showed tremendous potential for use in the CTSA context.

Conclusion: Despite challenges in making cross-CTSA comparisons, bibliometrics can enhance our understanding of the value of CTSA-supported clinical and translational research.

Keywords: CTSA; Metrics; Publications.

Fig. 1

Scholarly output and average cites per paper for Thomson Reuters (TR) and Elsevier: all publications (2007–2013).

Fig. 2

Thomson Reuters (TR) and Elsevier (E) scholarly output by institution and project year.

Fig. 3

Thomson Reuters Category-C Index (Cat-C Index), Elsevier Field-Weighted Citation Impact (FWCI), and iCite Relative Citation Ratio (RCR) by Year for all publications. The black line at 1.0 shows the expected rate for the respective comparative citation ratios.