A methodological review with meta-epidemiological analysis of preclinical systematic reviews with meta-analyses

Abstract

Systematic reviews and meta-analyses have been proposed as an approach to synthesize the literature and counteract the lack of power of small preclinical studies. We aimed to evaluate (1) the methodology of these reviews, (2) the methodological quality of the studies they included and (3) whether study methodological characteristics affect effect size. We searched MEDLINE to retrieve 212 systematic reviews with meta-analyses of preclinical studies published from January, 2018 to March, 2020. Less than 15% explored the grey literature. Selection, data extraction and risk of bias assessment were performed in duplicate in less than two thirds of reviews. Most of them assessed the methodological quality of included studies and reported the meta-analysis model. The risk of bias of included studies was mostly rated unclear. In meta-epidemiological analysis, none of the study methodological characteristics was associated with effect size. The methodological characteristics of systematic reviews with meta-analyses of recently published preclinical studies seem to have improved as compared with previous assessments, but the methodological quality of included studies remains poor, thus limiting the validity of their results. Our meta-epidemiological analysis did not show any evidence of a potential association between methodological characteristics of included studies and effect size.

Figure 1

Selection process of preclinical systematic reviews with meta-analyses. Adapted from the PRISMA flow diagram.

Figure 2

Difference in standardized mean difference (SMD) for risk of bias estimated by a meta-epidemiological analysis. A positive difference in SMD reveals a larger effect size in studies at high or unclear risk of bias. A negative difference in SMD indicates a smaller effect size in case of threats to methodological quality. Het heterogeneity, MA meta-analyses.