Homophily and missing links in citation networks

Abstract

Citation networks have been widely used to study the evolution of science through the lenses of the underlying patterns of knowledge flows among academic papers, authors, research sub-fields, and scientific journals. Here we focus on citation networks to cast light on the salience of homophily, namely the principle that similarity breeds connection, for knowledge transfer between papers. To this end, we assess the degree to which citations tend to occur between papers that are concerned with seemingly related topics or research problems. Drawing on a large data set of articles published in the journals of the American Physical Society between 1893 and 2009, we propose a novel method for measuring the similarity between articles through the statistical validation of the overlap between their bibliographies. Results suggest that the probability of a citation made by one article to another is indeed an increasing function of the similarity between the two articles. Our study also enables us to uncover missing citations between pairs of highly related articles, and may thus help identify barriers to effective knowledge flows. By quantifying the proportion of missing citations, we conduct a comparative assessment of distinct journals and research sub-fields in terms of their ability to facilitate or impede the dissemination of knowledge. Findings indicate that Electromagnetism and Interdisciplinary Physics are the two sub-fields in physics with the smallest percentage of missing citations. Moreover, knowledge transfer seems to be more effectively facilitated by journals of wide visibility, such as Physical Review Letters, than by lower-impact ones. Our study has important implications for authors, editors and reviewers of scientific journals, as well as public preprint repositories, as it provides a procedure for recommending relevant yet missing references and properly integrating bibliographies of papers.

Keywords: bibliometric techniques; citation networks; homophily; link prediction.

Figure 1

Quantifying the similarity between two articles based on their bibliographies. The similarity between two articles can be defined in terms of the overlap between their reference lists. The two articles P1 and P2 in panel (a) share only one citation; they should therefore be considered less similar than articles P3 and P4 in panel (b) which share four citations. This difference can be captured by the Jaccard index, which is equal to 0.2 in the former case and to 1.0 in the latter. However, the Jaccard index is equal to 1.0 also for the two articles in panel (c), which instead share only two citations. If citations are interpreted as proxies for knowledge flows, then the similarity between articles P7 and P8 in panel (d), which cite a highly-cited article, should be smaller than the similarity between articles P9 and P10 in panel (e), which instead are the only two articles citing P11. Our similarity measure, based on statistical validation, properly takes these heterogeneities into account.

Figure 2

The probability ${\mathit{P}}_{\mathit{i}\mathbf{\to }\mathit{j}}\mathbf{(}{\mathit{p}}^{\mathbf{\ast }}\mathbf{)}$ to observe a citation between two articles whose bibliographies overlap is statistically significant at the threshold value ${\mathit{p}}^{\mathbf{\ast }}$ . Notice that $P_{i\to j}(p^{\ast })$ increases as the statistical threshold $p^{\ast }$ decreases. That is, citations between pairs of articles characterised by a highly significant overlap tend to occur with a higher likelihood than citations between articles whose reference lists are not significantly similar. The inset shows how the number of pairs of articles characterised by a statistically significant similarity at a given threshold $p^{\ast }$ varies with $p^{\ast }$.

Figure 3

Lack of knowledge flows. An example of several validated pairs of articles in the APS citation network at $p^{\ast }={10}^{-7}$. Articles are reported in order of publication time, from older (left) to more recent (right) ones. The occurrence of a link indicates that the pair of articles has passed the statistical test, while the colour of the link indicates that the most recent article in the pair actually did (green) or did not (red) cite the other one. In this case, all the articles represented as yellow nodes are articles co-authored by researchers in the same group, while article A was co-authored by another group. The identification of a large number of missing citations suggests that the two groups might have been unaware of the work of their colleagues in the same field.

Figure 4

Ranking journals and sub-fields by lack of knowledge flows. The analysis of missing links restricted to specific sub-fields of physics or single APS journals confirms that the tendency of a citation to occur between a pair of articles increases with the similarity between the bibliographies of the two articles. Panels (a)-(b) show the plots of $U(p^{\ast })=1-P_{i\to j}(p^{\ast })$ for different sub-graphs corresponding to (a) two families of PACS codes, namely 40 (electromagnetism) and 50 (Gases and Plasmas), and (b) two APS journals, namely Physical Review Letters and Physical Review C. In panel (c) we sketch the procedure adopted to compute the estimate ${\tilde{U}}_0$: we consider the line tangent to the curve $U(p^{\ast })$ at the smallest value of the statistical threshold $p^{\ast }$ for which we still have a relatively substantial number of validated pairs (in this case, $p^{\ast }={10}^{-7}$), and we define ${\tilde{U}}_0$ as the value of the intercept at $p^{\ast }=0$ of that line. In panels (d) and (e) we show, respectively, the rankings of sub-fields and APS journals based on the values of ${\tilde{U}}_0$. Notice that Electromagnetism and Interdisciplinary physics are the two sub-fields with the smallest percentage of missing links, i.e., those in which knowledge among articles flows effectively and as would be expected if citations were driven by overlaps between topics or research problems. Interestingly, the lack of knowledge flows between articles published in Physical Review C (${\tilde{U}}_0\simeq 0.27$) is almost nine times as large as the one identified in Physical Review Letters (${\tilde{U}}_0\simeq 0.03$), which is the APS journal with the widest visibility and largest impact.