Fifty years of research on questionable research practises in science: quantitative analysis of co-citation patterns

Abstract

Questionable research practises (QRPs) have been the focus of the scientific community amid greater scrutiny and evidence highlighting issues with replicability across many fields of science. To capture the most impactful publications and the main thematic domains in the literature on QRPs, this study uses a document co-citation analysis. The analysis was conducted on a sample of 341 documents that covered the past 50 years of research in QRPs. Nine major thematic clusters emerged. Statistical reporting and statistical power emerged as key areas of research, where systemic-level factors in how research is conducted are consistently raised as the precipitating factors for QRPs. There is also an encouraging shift in the focus of research into open science practises designed to address engagement in QRPs. Such a shift is indicative of the growing momentum of the open science movement, and more research can be conducted on how these practises are employed on the ground and how their uptake by researchers can be further promoted. However, the results suggest that, while pre-registration and registered reports receive the most research interest, less attention has been paid to other open science practises (e.g. data sharing).

Keywords: ethics of research; questionable research practises; scientific integrity.

Figure 1.

Preferred reporting items for systematic reviews and meta-analyses (PRISMA) flowchart for literature search and reference eligibility.

Figure 2.

Top 50 keywords co-occurrences.

Figure 3.

Document co-citation analysis network of all literature on questionable research practises (QRPs). The network is created using graph theory principles, which allows the representation of the relationship between items through the use of edges and nodes. The graph represents the existing relationships in the literature on QRPs by using documents as nodes and co-citation patterns as edges. The nine major clusters are grouped by colour. The image was generated with the CiteSpace software [21].