Formulation of a CITE metric for evaluating the clinical implications of medical studies and their originating hospitals in China

Abstract

Background: The number of medical research publications by Chinese clinical investigators has risen substantially, contributing to 14.63% of the global total in 2019; however, their tangible impact on clinical decision-making remains limited. Various evaluation methods have been developed to measure hospital research competence in China, such as Fudan University's China hospital ranking and Science and Technology Evaluation Metrics (STEM) ranking, which predominantly focuses on factors such as academic reputation, volume of publications and patents, and research resources. However, composite indices may not fully capture the actual clinical value generated by medical research. To address this gap, we introduced the "Clinical Influence and Timeliness Evaluation (CITE)" metric to assess both the clinical importance of a given medical research study and the clinical influence of the hospital where it originated. The methodology used relies on the premise that influential medical research would be referenced in clinical guidelines, which serve as critical resources for clinicians.

Methods: The CITE metric was applied for 78,636 medical studies concerning chronic obstructive pulmonary disease (COPD) published between 2000 and 2020 and referenced in both Chinese and international clinical guidelines for COPD. Specific indexes and formulas were derived to quantify the clinical weight of a medical research study (W) and its timeliness (T), enabling a dynamic assessment of the clinical value of each study and the overall contribution of a particular hospital.

Results: In this analysis, we incorporated 499 hospitals in China and quantitatively identified their dynamic clinical influence in COPD from 2000 to 2020. Our findings offer objective and targeted evaluation metrics by focusing on clinical relevance and recognizing the collaborative nature of medical research.

Conclusion: The CITE metric provides an innovative method to gauge the true impact of medical research in China, with potential applications across different medical specialties. CITE can serve as a useful tool for understanding the relationship between research input and practical clinical outcomes, ultimately promoting more clinically relevant research endeavors.

Keywords: Chinese clinical investigations; Clinical Influence and Timeliness Evaluation (CITE); hospital research competence.

Figure 1

Medical research source analyzed in this study. (a) The medical research source comprises medical studies cited in the Chinese COPD Clinical Guidelines and the UpToDate topics (primary studies), as well as medical studies cited in these primary studies (secondary studies). (b) Number of primary (darker bars) and secondary (lighter bars) studies in the years from 2000 to 2020; the color signifies the data source origin as shown in (a). (c) Changing proportion of medical papers originating from China within the medical research source from 2000 to 2020. COPD, chronic obstructive pulmonary disease.

Figure 2

Schematic representation of method for calculating the clinical importance of medical studies (CIMS) and contribution value of author affiliations (CVAA) in this study.