A principal component analysis of 39 scientific impact measures

Abstract

Background: The impact of scientific publications has traditionally been expressed in terms of citation counts. However, scientific activity has moved online over the past decade. To better capture scientific impact in the digital era, a variety of new impact measures has been proposed on the basis of social network analysis and usage log data. Here we investigate how these new measures relate to each other, and how accurately and completely they express scientific impact.

Methodology: We performed a principal component analysis of the rankings produced by 39 existing and proposed measures of scholarly impact that were calculated on the basis of both citation and usage log data.

Conclusions: Our results indicate that the notion of scientific impact is a multi-dimensional construct that can not be adequately measured by any single indicator, although some measures are more suitable than others. The commonly used citation Impact Factor is not positioned at the core of this construct, but at its periphery, and should thus be used with caution.

Figure 1. Schematic representation of data sources and processing.

Impact measure identifiers refer to Table 1.

Figure 2. Correlations between 37 measures mapped onto first two principal components (cumulative variance = 83.4%) of PCA.

Black dots indicate citation-based measures. White dots indicate usage-based measures. The Journal Impact Factor (5) has a blue lining. Measures 23 and 39 excluded.

Figure 3. Hierarchical cluster analysis of 39 impact measures (excluding measures 23 and 39).

Figure 4. Schematic representation of PCA analysis shown in Fig. 2.