How citation distortions create unfounded authority: analysis of a citation network

Abstract

Objective: To understand belief in a specific scientific claim by studying the pattern of citations among papers stating it.

Design: A complete citation network was constructed from all PubMed indexed English literature papers addressing the belief that beta amyloid, a protein accumulated in the brain in Alzheimer's disease, is produced by and injures skeletal muscle of patients with inclusion body myositis. Social network theory and graph theory were used to analyse this network.

Main outcome measures: Citation bias, amplification, and invention, and their effects on determining authority.

Results: The network contained 242 papers and 675 citations addressing the belief, with 220,553 citation paths supporting it. Unfounded authority was established by citation bias against papers that refuted or weakened the belief; amplification, the marked expansion of the belief system by papers presenting no data addressing it; and forms of invention such as the conversion of hypothesis into fact through citation alone. Extension of this network into text within grants funded by the National Institutes of Health and obtained through the Freedom of Information Act showed the same phenomena present and sometimes used to justify requests for funding.

Conclusion: Citation is both an impartial scholarly method and a powerful form of social communication. Through distortions in its social use that include bias, amplification, and invention, citation can be used to generate information cascades resulting in unfounded authority of claims. Construction and analysis of a claim specific citation network may clarify the nature of a published belief system and expose distorted methods of social citation.

Fig 1 Claim specific citation network. Citations regarding claim that β amyloid precursor protein mRNA or protein, or β amyloid protein, is abnormally present in inclusion body myositis muscle. The network is organised according to paper category and year of publication. Authority status (yellow) was defined computationally by network theory. Many citations flow to supportive primary data but not critical data. Papers are represented as nodes (n=218) and citations as directed edges (supportive n=636, neutral n=18, critical n=21, diversion n=3). Twenty four papers contain statements pertaining to claim but do not make or receive citations about it (not shown)

Fig 2 Citation bias against content critical of claim. Shown are citation frequencies to four authoritative supportive primary data papers and six primary data papers containing data critical of claim

Fig 3 Citations from animal and cell culture model papers to primary data papers supporting rationale for overproduction of β amyloid precursor protein mRNA as a valid model of inclusion body myositis. Only one of 32 citations flows to papers that present data that conflict with the validity of these models

Fig 4 Amplification and authority of claim. (Left) Historical growth of supportive and critical citations in network. (Middle) Rearrangement of network (fig 1) to visualise a lens effect in which eight key papers (surrounded by ovals; seven by same research group; 97% of all network traffic passes through them) create citation flows among each other, and both amplify claim and focus citations to supportive data papers. Net effect results in network authority status (yellow) of supportive data papers. (Right) Computational elimination of citation bias results in balanced authority of both support for claim and its refutation through additional recognition of critical data papers as authorities

Fig 5 Conversion of hypothesis to fact through citation alone. Citations on statement that accumulation of β amyloid “precedes” other abnormalities in inclusion body myositis muscle. Statement as fact is supported through citation to papers that only state it as hypothesis (for example, references 5 to 80, 91 to 80, 134 to 74) or sometimes supported by citation to papers that contain no statements addressing it (for example, references 91 to 72, 251 to 75; dead end citations). This phenomenon might be called citation transmutation (see web extra note 10 for statements)

Fig 6 Extension of PubMed claim specific citation network into National Institutes of Health funded research proposals. Nine funded research grants (G1-G9; see web extra note 13) contain statements and citations addressing claim; their citations to primary data are shown. Citation bias and citation diversion are present

Fig 7 Overview of approach. After construction of the claim specific citation network, a combined manual and computational endeavour, steps on left (determination of authorities [yellow papers] and identification of amplification) require only computational algorithms; right half (identifying which papers contain actual data addressing claim validity and identifying invention) requires careful reading of paper content. Combining results of authority identification with data identification allows for recognition of citation bias and subsequent steps for its simulated removal and assessment of effects on network