Checklists of methodological issues for review authors to consider when including non-randomized studies in systematic reviews

Abstract

Background: There is increasing interest from review authors about including non-randomized studies (NRS) in their systematic reviews of health care interventions. This series from the Ottawa Non-Randomized Studies Workshop consists of six papers identifying methodological issues when doing this.

Aim: To format the guidance from the preceding papers on study design and bias, confounding and meta-analysis, selective reporting, and applicability/directness into checklists of issues for review authors to consider when including NRS in a systematic review.

Checklists: Checklists were devised providing frameworks to describe/assess: (1) study designs based on study design features; (2) risk of residual confounding and when to consider meta-analysing data from NRS; (3) risk of selective reporting based on the Cochrane framework for detecting selective outcome reporting in trials but extended to selective reporting of analyses; and (4) directness of evidence contributed by a study to aid integration of NRS findings into summary of findings tables.

Summary: The checklists described will allow review groups to operationalize the inclusion of NRS in systematic reviews in a more consistent way. The next major step is extending the existing Cochrane Risk of Bias tool so that it can assess the risk of bias to NRS included in a review. Copyright © 2013 John Wiley & Sons, Ltd.

Keywords: checklists; methodological issues; non-randomized studies; systematic review.