Methodological quality for systematic reviews of adverse events with surgical interventions: a cross-sectional survey

Abstract

Background: An increasing number of systematic reviews assessed the safety of surgical interventions over time. How well these systematic reviews were designed and conducted determines the reliability of evidence. In this study, we aimed to assess the methodological quality of systematic reviews on the safety of surgical interventions.

Methods: We searched PubMed for systematic reviews of surgical interventions with safety as the exclusive outcome from 1st-Jan, 2015 to 1st-Jan, 2020. The methodological quality of eligible systematic reviews was evaluated according to the AMSTAR 2.0 instrument. The primary outcomes were the number of methodological weaknesses and the global methodological quality. The proportion of each methodological weakness among eligible systematic reviews was compared by three pre-defined stratification variables. The absolute difference of the proportion (PD) was used as the effect estimator, with the two-tailed z-test for the significance.

Results: We identified 127 systematic reviews from 18,636 records. None (n = 0, 0.00%) of them could be rated as "high" in terms of the global methodological quality; in contrast, they were either rated as "low" (n = 18, 14.17%) or as "critically low" (n = 109, 85.83%). The median number of methodological weaknesses of these systematic reviews was 8 (interquartile range, IQR: 6 to 9), in which 4 (IQR: 2 to 4) were critical weaknesses. Systematic reviews that used any reporting guideline (e.g., domain 13, PD = -0.22, 95% CI: - 0.39, - 0.06; p = 0.01) and developed a protocol in advance (e.g., domain 6, PD = -0.20, 95% CI: - 0.39, - 0.01; p = 0.04) were less likely to have methodological weakness in some domains but not for the rest (e.g., domain 8, PD = 0.04, 95% CI: - 0.14, 0.21; p = 0.68; with protocol vs. without).

Conclusions: The methodological quality of current systematic reviews of adverse events with surgical interventions was poor. Further efforts, for example, encouraging researchers to develop a protocol in advance, are needed to enhance the methodological quality of these systematic reviews.

Keywords: Adverse event; Methodological quality; Surgical intervention; Systematic review.

Fig. 1

Flow plot of literature search and screening

Fig. 2

Methodological quality of 127 systematic reviews. Domains 11, 12, and 15 do not include the 31 qualitative systematic reviews without meta-analyses

Fig. 3

Distribution of methodological weaknesses. SR: systematic review

Fig. 4

Factors that may relate to the methodological weaknesses. Notes: The domains with absolute PDs larger than 0.1 (in absolute magnitude) and/or with p-values less than 0.05 were highlighted in bold. The variable development of protocol is exactly the domain 2 in AMSTAR 2.0 instrument. Therefore, we did not compare the rate of “No” for item 2 by this variable. Similarly, domains 11, 12, and 15 were not suitable for qualitative systematic reviews without meta-analysis