Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data

Abstract

Objectives: The aim of the study was to assess the completeness of reporting of methods in overviews.

Study design and setting: Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented.

Results: We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7-32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]).

Conclusion: Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.

Keywords: Meta-review; Overview methods; Overviews of systematic reviews; Reporting; Systematic review methods; Umbrella.