Analysis of the systematic reviews process in reports of network meta-analyses: methodological systematic review

Abstract

Objective: To examine whether network meta-analyses, increasingly used to assess comparative effectiveness of healthcare interventions, follow the key methodological recommendations for reporting and conduct of systematic reviews.

Design: Methodological systematic review of reports of network meta-analyses.

Data sources: Cochrane Database of Systematic Reviews, Database of Abstracts of Reviews of Effects, Medline, and Embase, searched from inception to 12 July 2012.

Review methods: All network meta-analyses comparing clinical efficacy of three or more interventions based on randomised controlled trials, excluding meta-analyses with an open loop network of three interventions. We assessed the reporting of general characteristics and key methodological components of the systematic review process using two composite outcomes. For some components, if reporting was adequate, we assessed their conduct quality.

Results: Of 121 network meta-analyses covering a wide range of medical areas, 100 (83%) assessed pharmacological interventions and 11 (9%) non-pharmacological interventions; 56 (46%) were published in journals with a high impact factor. The electronic search strategy for each database was not reported in 88 (73%) network meta-analyses; for 36 (30%), the primary outcome was not clearly identified. Overall, 61 (50%) network meta-analyses did not report any information regarding the assessment of risk of bias of individual studies, and 103 (85%) did not report any methods to assess the likelihood of publication bias. Overall, 87 (72%) network meta-analyses did not report the literature search, searched only one database, did not search other sources, or did not report an assessment of risk of bias of individual studies. These methodological components did not differ by publication in a general or specialty journal or by public or private funding.

Conclusions: Essential methodological components of the systematic review process-conducting a literature search and assessing risk of bias of individual studies-are frequently lacking in reports of network meta-analyses, even when published in journals with high impact factors.

Fig 1 Flowchart of selection of network meta-analyses. CDSR=Cochrane Database of Systematic Reviews; DARE= Database of Abstracts of Reviews of Effects; NMA=network meta-analysis; RCT= randomised controlled trial

Fig 2 Number of reports of network meta-analyses published per year. *We estimated the number of reports published from July to December 2012 on the basis of the number of published network meta-analyses from the six previous months. Between January and June 2012, 27 reports were published (dark region); therefore, we estimated 27 reports from July to December 2012 (light region)