A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments

Abstract

Objectives: To review the literature and obtain preferences and perceptions from experts regarding the role of randomized studies (RSs) and nonrandomized studies (NRSs) in systematic reviews of intervention effects.

Study design and setting: Scoping review and survey of experts. Using levels of certainty developed by the Grading of Recommendations Assessment, Development and Evaluation (GRADE) working group, experts expressed their preferences about the use of RS and NRS in health syntheses.

Results: Of 189 respondents, 123 had the expertise required to answer the questionnaire; 116 provided their extent of agreement with approaches to use NRS with RS. Most respondents would include NRS when RS was unfeasible (83.6%) or unethical (71.5%) and a majority to maximize the body of evidence (66.3%), compare results in NRS and RS (53.5%) and to identify subgroups (51.7%). Sizable minorities would include NRS and RS to address the effect of randomization (29.5%) or because the question being addressed was a public-health intervention (36.5%). In summary of findings tables, most respondents would include both bodies of evidence-in two rows in the same table-when RS provided moderate, low, or very-low certainty evidence; even when RS provided high certainty evidence, a sizable minority (25%) would still present results from both bodies of evidence. Very few (3.6%) would, under realistic circumstances, pool RS and NRS results.

Conclusions: Most experts would include both RS and NRS in the same review under a wide variety of circumstances, but almost all would present results of two bodies of evidence separately.

Keywords: Clinical practice guidelines; GRADE; Nonrandomized trials; Randomized trials; Research methodology; Systematic reviews.