Quality, topics, and demographic trends of animal systematic reviews - an umbrella review

Abstract

Background: Animal systematic reviews are critical to inform translational research. Despite their growing popularity, there is a notable lack of information on their quality, scope, and geographical distribution over time. Addressing this gap is important to maintain their effectiveness in fostering medical advancements.

Objective: This study aimed to assess the quality and demographic trends of animal systematic reviews in neuroscience, including changes over time.

Methods: We performed an umbrella review of animal systematic reviews, searching Medline and Embase for reviews until January 27, 2023. A data mining method was developed and validated to automatically evaluate the quality of these reviews.

Results: From 18'065 records identified, we included 1'358 animal systematic reviews in our study. These reviews commonly focus on translational research but with notable topical gaps such as schizophrenia, other psychiatric disorders, and brain tumours. They originate from 64 countries, with the United States, China, the UK, Brazil, and Iran being the most prolific. The automated quality assessment indicated high reliability, with F1-scores over 80% for most criteria. Overall, the reviews were of high quality and the quality improved over time. However, many systematic reviews did not report a pre-registered study protocol. Reviews with a pre-registered protocol generally scored higher in quality. No significant differences in quality were observed between countries.

Conclusion: Animal systematic reviews in neuroscience are of overall of high quality. Our study highlights specific areas for enhancement such as the recommended pre-publication of study protocols. It also identifies under-represented topics that could benefit from further investigation to inform translational research. Such measures can contribute to the effective translation of animal research findings to clinical applications.

Keywords: Animal research; Animal welfare; Automation; Evidence map; Neuroscience; Risk of bias; Systematic review; Translational research.

Fig. 1

Flow chart for study inclusion

Fig. 2

Topics addressed by animal systematic reviews. Word cloud of covered topics (A) and topic quantification per research domain (B)

Fig. 3

Disease burden metrics and coverage of preclinical systematic reviews. This table presents disease burden metrics, including global disease prevalence (in thousands), disability-adjusted life-years (DALY), years lived with disability (YLD), years of life lost (YLL), and deaths. The metrics are based on disease categories from the Global Burden of Disease study for neurological [28] and mental disorders [29]. Additionally, the table includes the number of systematic reviews per disease, further categorized by focal topics: therapeutic interventions, pathophysiology and mechanisms, diagnostic tools, and biomarkers, and other. The first column displays the ratio of systematic reviews to disease prevalence (multiplied by 10,000) to indicate the relative over- or under-representation of systematic reviews for each disease. Abbreviations: DALYs = disability-adjusted life-years; SR, systematic review; YLDs = years lived with disability; YLLs = years of life lost

Fig. 4

World heat map of countries publishing animal systematic reviews. Animal systematic reviews stemmed from 64 countries across all continents

Fig. 5

Prolific countries publishing animal systematic reviews. The 5 most prolific countries were the USA, China, UK, Brazil, and Iran (A). Europe was the most prolific continent in the production of systematic review (B). The median global growth of systematic reviews was 26%

Fig. 6

Quality of animal systematic reviews. Quality of animal systematic reviews overall (A), per scored item (B), and for the most prolific countries publishing animal systematic reviews (C). There is a significant increase in systematic review quality over time. The countries show no statistically significant quality differences