Day-to-day discovery of preprint-publication links

Abstract

Preprints promote the open and fast communication of non-peer reviewed work. Once a preprint is published in a peer-reviewed venue, the preprint server updates its web page: a prominent hyperlink leading to the newly published work is added. Linking preprints to publications is of utmost importance as it provides readers with the latest version of a now certified work. Yet leading preprint servers fail to identify all existing preprint-publication links. This limitation calls for a more thorough approach to this critical information retrieval task: overlooking published evidence translates into partial and even inaccurate systematic reviews on health-related issues, for instance. We designed an algorithm leveraging the Crossref public and free source of bibliographic metadata to comb the literature for preprint-publication links. We tested it on a reference preprint set identified and curated for a living systematic review on interventions for preventing and treating COVID-19 performed by international collaboration: the COVID-NMA initiative (covid-nma.com). The reference set comprised 343 preprints, 121 of which appeared as a publication in a peer-reviewed journal. While the preprint servers identified 39.7% of the preprint-publication links, our linker identified 90.9% of the expected links with no clues taken from the preprint servers. The accuracy of the proposed linker is 91.5% on this reference set, with 90.9% sensitivity and 91.9% specificity. This is a 16.26% increase in accuracy compared to that of preprint servers. We release this software as supplementary material to foster its integration into preprint servers' workflows and enhance a daily preprint-publication chase that is useful to all readers, including systematic reviewers. This preprint-publication linker currently provides day-to-day updates to the biomedical experts of the COVID-NMA initiative.

Keywords: COVID-19; Data linking; Living systematic review; Preprint; Publication.

Fig. 1

Crossref documentation on preprint metadata updates expected from content publishers. Excerpt of the Introduction to posted content (including preprints) available from https://www.crossref.org/education/content-registration/content-types-intro/posted-content-includes-preprints/

Fig. 2

The medRxiv preprint doi:10.1101/2020.04.07.20056424 with linked paper in JAMA Network Open

Fig. 3

Excerpt of the listing of all medRxiv preprints, available from https://api.biorxiv.org/details/medrxiv/2000-01-01/3000-01-01/1 in JSON format. Each preprint comes with its associated metadata (e.g., title, authors, version) and is optionally linked to a publication (e.g., see the third record with 10.1136/bmj.l7078)

Fig. 4

Validated preprint–publication links shown on the ‘COVID19 Preprint Tracker’ used by COVID-NMA and hosted at https://www.irit.fr/~Guillaume.Cabanac/covid19-preprint-tracker

Fig. 5

Evaluation of the preprint servers on the 343 preprint reference set, as of 23 October 2020

Fig. 6

Evaluation of the preprint–publication linker on the 343 preprint reference set, as of 23 October 2020