General medical publications during COVID-19 show increased dissemination despite lower validation

Abstract

Background: The COVID-19 pandemic has yielded an unprecedented quantity of new publications, contributing to an overwhelming quantity of information and leading to the rapid dissemination of less stringently validated information. Yet, a formal analysis of how the medical literature has changed during the pandemic is lacking. In this analysis, we aimed to quantify how scientific publications changed at the outset of the COVID-19 pandemic.

Methods: We performed a cross-sectional bibliometric study of published studies in four high-impact medical journals to identify differences in the characteristics of COVID-19 related publications compared to non-pandemic studies. Original investigations related to SARS-CoV-2 and COVID-19 published in March and April 2020 were identified and compared to non-COVID-19 research publications over the same two-month period in 2019 and 2020. Extracted data included publication characteristics, study characteristics, author characteristics, and impact metrics. Our primary measure was principal component analysis (PCA) of publication characteristics and impact metrics across groups.

Results: We identified 402 publications that met inclusion criteria: 76 were related to COVID-19; 154 and 172 were non-COVID publications over the same period in 2020 and 2019, respectively. PCA utilizing the collected bibliometric data revealed segregation of the COVID-19 literature subset from both groups of non-COVID literature (2019 and 2020). COVID-19 publications were more likely to describe prospective observational (31.6%) or case series (41.8%) studies without industry funding as compared with non-COVID articles, which were represented primarily by randomized controlled trials (32.5% and 36.6% in the non-COVID literature from 2020 and 2019, respectively).

Conclusions: In this cross-sectional study of publications in four general medical journals, COVID-related articles were significantly different from non-COVID articles based on article characteristics and impact metrics. COVID-related studies were generally shorter articles reporting observational studies with less literature cited and fewer study sites, suggestive of more limited scientific support. They nevertheless had much higher dissemination.

Fig 1. Principal component analysis of COVID and non-COVID publication characteristics and impact metrics.

Each point in the plot corresponds to a single characteristic provided in Table 1 for COVID (green square) and non-COVID publications from 2019 (purple circle) and 2020 (gray triangle). Principal component 1 (PC1) is shown plotted against (A) PC2 and (B) PC3. PC1, PC2, and PC3 respectively account for 32.4%, 24.8% and 16.4% of the variability. Non-COVID publications from 2019 and 2020 clusters overlap, whereas COVID publications cluster separately. This unbiased analysis suggests COVID-related publications differ from both concurrent and historic non-COVID publications.