Several methods for assessing research waste in reviews with a systematic search: a scoping review

Abstract

Background: Research waste is present in all study designs and can have significant consequences for science, including reducing the reliability of research findings and contributing to the inefficient use of resources. Estimates suggest that as much as 85% of all biomedical research is wasted. However, it is uncertain how avoidable research waste is assessed in specific types of study designs and what methods could be used to examine different aspects of research waste. We aimed to investigate which methods, systematic reviews, scoping reviews, and overviews of reviews discussing research waste, have used to assess avoidable research waste.

Materials and methods: We published a protocol in the Open Science Framework prospectively (https://osf.io/2fbp4). We searched PubMed and Embase with a 30-year limit (January 1993-August 2023). The concept examined was how research waste and related synonyms (e.g., unnecessary, redundant, duplicate, etc.) were assessed in reviews with a systematic search: systematic, scoping, or overviews of reviews. We extracted data on the method used in the review to examine for research waste and for which study design this method was applied.

Results: The search identified 4,285 records of which 93 reviews with systematic searches were included. The reviews examined a median of 90 (range 10-6,781) studies, where the study designs most commonly included were randomized controlled trials (48%) and systematic reviews (33%). In the last ten years, the number of reports assessing research waste has increased. More than 50% of examined reviews reported evaluating methodological research waste among included studies, typically using tools such as one of Cochrane Risk of Bias tools (n = 8) for randomized controlled trials or AMSTAR 1 or 2 (n = 12) for systematic reviews. One fourth of reviews assessed reporting guideline adherence to e.g., CONSORT (n = 4) for randomized controlled trials or PRISMA (n = 6) for systematic reviews.

Conclusion: Reviews with systematic searches focus on methodological quality and reporting guideline adherence when examining research waste. However, this scoping review revealed that a wide range of tools are used, which may pose difficulties in comparing examinations and performing meta-research. This review aids researchers in selecting methodologies and contributes to the ongoing discourse on optimizing research efficiency.

Keywords: Duplicate efforts; Evidence-based medicine; Redundant research; Research waste; Scoping review; Systematic review.

Figure 1. PRISMA flow diagram depicting the selection of reports in the scoping review.

(A) Research misconduct involves intentional violations of ethical standards and integrity, fundamentally different from research waste, e.g., inefficiencies and inadequacies in the research process that lead to the loss of valuable resources, duplication of effort, and reduced reliability of findings. (B) Reported elsewhere (Rosengaard et al., 2024b).

Figure 2. An UpSet plot (Greenhalgh & Peacock, 2005) of the distribution of the examined aspect of research waste according to MINUS and the overlap of aspects examined by the included reviews (n = 93).

Figure 3. Treemap for each of the examined five aspects of research waste MINUS (Rosengaard et al., 2024b) and what the reviews investigated within each aspect.