Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews

Affiliations

¹ Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA. Electronic address: swieland@som.umaryland.edu.
² Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA.
³ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, Botnar Research Centre, Windmill Road, University of Oxford, Oxford, OX3 7LD, UK.
⁴ Institute for Complementary and Integrative Medicine, University Hospital Zurich, University of Zürich, Rämistrasse 100, Zürich CH-8091, Switzerland.
⁵ Department of Epidemiology & Biostatistics, VU University Medical Center, PO Box 7057, Amsterdam, 1007 MB, The Netherlands; Department of Philosophy, Faculty of Humanities, Vrije Universiteit, De Boelelaan 1105, Amsterdam 1081 HV, The Netherlands.
⁶ Institute of General Practice, University Hospital rechts der Isar, Technical University Munich, Orleansstrasse 47, Munich 81667, Germany.
⁷ Clinical Epidemiology Program, Ottawa Hospital Research Institute, 725 Parkdale Avenue, Ottawa, ON K1Y 4E9, Canada; School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, 451 Smyth Road, Ottawa ON K1H 8M5, Canada.
⁸ Pharmaceutical Health Services Research Department, University of Maryland School of Pharmacy, 20 North Pine Street, Baltimore, MD 21201, USA.
⁹ Health Services Research Unit, University of Aberdeen, 3rd Floor, Health Sciences Building, Foresterhill, Aberdeen AB25 2ZD, UK.
¹⁰ Center for Medical Technology Policy, World Trade Center Institute, 401 E Pratt St # 631, Baltimore, MD 21202, USA.
¹¹ Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Primary Care Sciences, Keele University, Staffordshire ST5 5BG, UK.
¹² Centre for Studies in Family Medicine, Schulich School of Medicine & Dentistry, Western University, Western Centre for Public Health and Family Medicine, 1151 Richmond Street, London, ON, N6A 3K7, Canada.
¹³ Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA; Institute for Complementary and Integrative Medicine, University Hospital Zurich, University of Zürich, Rämistrasse 100, Zürich CH-8091, Switzerland; Institute for Social Medicine, Epidemiology and Health Economics, Charité Universitätsmedizin Berlin, Luisenstr. 57, Berlin 10117, Germany.

PMID: 28188898
PMCID: PMC5441969
DOI: 10.1016/j.jclinepi.2017.01.010

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews

L Susan Wieland et al. J Clin Epidemiol. 2017 Apr.

. 2017 Apr:84:95-104.

doi: 10.1016/j.jclinepi.2017.01.010. Epub 2017 Feb 7.

Authors

Affiliations

¹ Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA. Electronic address: swieland@som.umaryland.edu.
² Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA.
³ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, Botnar Research Centre, Windmill Road, University of Oxford, Oxford, OX3 7LD, UK.
⁴ Institute for Complementary and Integrative Medicine, University Hospital Zurich, University of Zürich, Rämistrasse 100, Zürich CH-8091, Switzerland.
⁵ Department of Epidemiology & Biostatistics, VU University Medical Center, PO Box 7057, Amsterdam, 1007 MB, The Netherlands; Department of Philosophy, Faculty of Humanities, Vrije Universiteit, De Boelelaan 1105, Amsterdam 1081 HV, The Netherlands.
⁶ Institute of General Practice, University Hospital rechts der Isar, Technical University Munich, Orleansstrasse 47, Munich 81667, Germany.
⁷ Clinical Epidemiology Program, Ottawa Hospital Research Institute, 725 Parkdale Avenue, Ottawa, ON K1Y 4E9, Canada; School of Epidemiology, Public Health and Preventive Medicine, University of Ottawa, 451 Smyth Road, Ottawa ON K1H 8M5, Canada.
⁸ Pharmaceutical Health Services Research Department, University of Maryland School of Pharmacy, 20 North Pine Street, Baltimore, MD 21201, USA.
⁹ Health Services Research Unit, University of Aberdeen, 3rd Floor, Health Sciences Building, Foresterhill, Aberdeen AB25 2ZD, UK.
¹⁰ Center for Medical Technology Policy, World Trade Center Institute, 401 E Pratt St # 631, Baltimore, MD 21202, USA.
¹¹ Arthritis Research UK Primary Care Centre, Research Institute for Primary Care & Health Sciences, Primary Care Sciences, Keele University, Staffordshire ST5 5BG, UK.
¹² Centre for Studies in Family Medicine, Schulich School of Medicine & Dentistry, Western University, Western Centre for Public Health and Family Medicine, 1151 Richmond Street, London, ON, N6A 3K7, Canada.
¹³ Center for Integrative Medicine, University of Maryland School of Medicine, 520 West Lombard Street, East Hall, Baltimore, MD, 21201, USA; Institute for Complementary and Integrative Medicine, University Hospital Zurich, University of Zürich, Rämistrasse 100, Zürich CH-8091, Switzerland; Institute for Social Medicine, Epidemiology and Health Economics, Charité Universitätsmedizin Berlin, Luisenstr. 57, Berlin 10117, Germany.

PMID: 28188898
PMCID: PMC5441969
DOI: 10.1016/j.jclinepi.2017.01.010

Abstract

Background and objective: Randomized trials may be designed to provide evidence more strongly related to efficacy or effectiveness of an intervention. When systematic reviews are used to inform clinical or policy decisions, it is important to know the efficacy-effectiveness nature of the included trials. The objective of this study was to develop a tool to characterize randomized trials included in a systematic review on an efficacy-effectiveness continuum.

Methods: We extracted rating domains and descriptors from existing tools and used a modified Delphi procedure to condense the domains and develop a new tool. The feasibility and interrater reliability of the tool was tested on trials from four systematic reviews.

Results: The Rating of Included Trials on the Efficacy-Effectiveness Spectrum (RITES) tool rates clinical trials on a five-point Likert scale in four domains: (1) participant characteristics, (2) trial setting, (3) flexibility of interventions, and (4) clinical relevance of interventions. When RITES was piloted on trials from three reviews by unaffiliated raters, ratings were variable (intraclass correlation coefficient [ICC] 0.25-0.66 for the four domains); but, when RITES was used on one review by the review authors with expertise on the topic, the ratings were consistent (ICCs > 0.80.

Conclusion: RITES may help to characterize the efficacy-effectiveness nature of trials included in systematic reviews.

Keywords: Applicability; Comparative effectiveness research; Effectiveness; Efficacy; Explanatory trial; Pragmatic trial; Randomized controlled trials; Systematic reviews.

PubMed Disclaimer

Figures

**Figure 1**
Visual presentation of the efficacy-effectiveness of trials within a systematic review

See this image and copyright information in PMC

References

1. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. Journal of clinical epidemiology. 2009;62:499–505. - PubMed
1. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Journal of clinical epidemiology. 2009;62:464–75. - PubMed
1. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. Bmj. 2015;350:h2147. - PubMed
1. Witt CM, Manheimer E, Hammerschlag R, Ludtke R, Lao L, Tunis SR, et al. How well do randomized trials inform decision making: systematic review using comparative effectiveness research measures on acupuncture for back pain. PloS one. 2012;7:e32399. - PMC - PubMed
1. Koppenaal T, Linmans J, Knottnerus JA, Spigt M. Pragmatic vs. explanatory: an adaptation of the PRECIS tool helps to judge the applicability of systematic reviews for daily practice. Journal of clinical epidemiology. 2011;64:1095–101. - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews

Affiliations

Rating of Included Trials on the Efficacy-Effectiveness Spectrum: development of a new tool for systematic reviews

Authors

Affiliations

Abstract

Figures

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources