This is a preprint.

It has not yet been peer reviewed by a journal.

The National Library of Medicine is running a pilot to include preprints that result from research funded by NIH in PMC and PubMed.

[Preprint]. 2023 Nov 13:2023.09.21.23295626.

doi: 10.1101/2023.09.21.23295626.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions

Jack Wilkinson¹, Calvin Heal¹, George A Antoniou^{2

3}, Ella Flemyng⁴, Zarko Alfirevic⁵, Alison Avenell⁶, Virginia Barbour⁷, Nicholas J L Brown⁸, John Carlisle⁹, Mike Clarke¹⁰, Patrick Dicker¹¹, Jo Dumville^{12

13}, Andrew Grey¹⁴, Steph Grohmann⁴, Lyle C Gurrin¹⁵, Jill A Hayden¹⁶, James Heathers¹⁷, Kylie E Hunter¹⁸, Toby Lasserson⁴, Emily Lam¹⁹, Sarah Lensen²⁰, Tianjing Li²¹, Wentao Li²², Elizabeth Loder^{23

24}, Andreas Lundh^{25

26}, Gideon Meyerowitz-Katz²⁷, Ben W Mol^{28

29}, Neil E O' Connell³⁰, Lisa Parker³¹, Barbara K Redman³², Anna Lene Seidler¹⁸, Kyle A Sheldrick³³, Emma Sydenham³⁴, David J Torgerson³⁵, Madelon van Wely^{36

37}, Rui Wang²², Lisa Bero³⁸, Jamie J Kirkham¹

Affiliations

¹ Centre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
² Manchester Vascular Centre, Manchester University NHS Foundation Trust, Manchester, UK.
³ Division of Cardiovascular Sciences, School of Medical Sciences, Manchester Academic Health Science Centre, The University of Manchester, Manchester, UK.
⁴ Evidence Production and Methods Directorate, Cochrane Central Executive, London, UK.
⁵ Emeritus Professor, University of Liverpool, UK.
⁶ Health Services Research Unit, University of Aberdeen, UK.
⁷ Medical Journal of Australia, Sydney, Australia.
⁸ Department of Psychology, Linnaeus University, Växjö, Sweden.
⁹ Perioperative Medicine, Torbay hospital, UK.
¹⁰ Department of Epidemiology and Public Health, Royal College of Surgeons in Ireland.
¹¹ Northern Ireland Methodology Hub, Queen's University Belfast, Northern Ireland.
¹² Division of Nursing, Midwifery & Social Work, School of Health Sciences, The University of Manchester, Manchester, UK.
¹³ NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK.
¹⁴ Department of Medicine, University of Auckland, Auckland, New Zealand.
¹⁵ School of Population and Global Health, The University of Melbourne, Australia.
¹⁶ Department of Community Health & Epidemiology, Faculty of Medicine, Dalhousie University, Canada.
¹⁷ SafeBeat Rx Inc, USA.
¹⁸ Evidence Integration, NHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia.
¹⁹ Independent lay member, unaffiliated, Cheshire, UK.
²⁰ Department of Obstetrics and Gynaecology, Royal Women's Hospital, University of Melbourne, Melbourne, Australia.
²¹ Department of Ophthalmology, University of Colorado Anschutz Medical Campus.
²² Department of Obstetrics and Gynaecology, Monash University, VIC, Australia.
²³ The BMJ, London, UK.
²⁴ Department of Neurology, Brigham and Women's Hospital, Boston, MA, USA.
²⁵ Cochrane Denmark & Centre for Evidence-Based Medicine Odense, Department of Clinical Research, University of Southern Denmark, Denmark.
²⁶ Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital, Bispebjerg and Frederiksberg, Denmark.
²⁷ School of Health and Society, University of Wollongong, Wollongong, Australia.
²⁸ Monash University, Clayton, Victoria, Australia.
²⁹ Aberdeen Centre for Women's Health Research, University of Aberdeen, Aberdeen, UK.
³⁰ Centre for Health and Wellbeing across the Lifecourse, Dept of Health Sciences, Brunel University London, UK.
³¹ School of Pharmacy, Charles Perkins Centre, The University of Sydney, Australia.
³² Division of Medical Ethics, New York University Grossman School of Medicine, New York, NY.
³³ Faculty of Medicine, University of New South Wales, Australia.
³⁴ Cochrane Central Production Service, UK.
³⁵ York Trials Unit, Dept of Health Sciences, University of York, UK.
³⁶ Cochrane Gynaecology and Fertility Satellite and Cochrane Sexually Transmitted Infection Group, Amsterdam.
³⁷ Reproduction and Development Research Institute, Amsterdam University Medical Center, The Netherlands.
³⁸ University of Colorado Anschutz Medical Campus, Colorado, USA.

PMID: 37873409
PMCID: PMC10593010
DOI: 10.1101/2023.09.21.23295626

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions

Jack Wilkinson et al. medRxiv. 2023.

[Preprint]. 2023 Nov 13:2023.09.21.23295626.

doi: 10.1101/2023.09.21.23295626.

Authors

Affiliations

¹ Centre for Biostatistics, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
² Manchester Vascular Centre, Manchester University NHS Foundation Trust, Manchester, UK.
³ Division of Cardiovascular Sciences, School of Medical Sciences, Manchester Academic Health Science Centre, The University of Manchester, Manchester, UK.
⁴ Evidence Production and Methods Directorate, Cochrane Central Executive, London, UK.
⁵ Emeritus Professor, University of Liverpool, UK.
⁶ Health Services Research Unit, University of Aberdeen, UK.
⁷ Medical Journal of Australia, Sydney, Australia.
⁸ Department of Psychology, Linnaeus University, Växjö, Sweden.
⁹ Perioperative Medicine, Torbay hospital, UK.
¹⁰ Department of Epidemiology and Public Health, Royal College of Surgeons in Ireland.
¹¹ Northern Ireland Methodology Hub, Queen's University Belfast, Northern Ireland.
¹² Division of Nursing, Midwifery & Social Work, School of Health Sciences, The University of Manchester, Manchester, UK.
¹³ NIHR Manchester Biomedical Research Centre, Manchester University NHS Foundation Trust, Manchester Academic Health Science Centre, Manchester, UK.
¹⁴ Department of Medicine, University of Auckland, Auckland, New Zealand.
¹⁵ School of Population and Global Health, The University of Melbourne, Australia.
¹⁶ Department of Community Health & Epidemiology, Faculty of Medicine, Dalhousie University, Canada.
¹⁷ SafeBeat Rx Inc, USA.
¹⁸ Evidence Integration, NHMRC Clinical Trials Centre, University of Sydney, Sydney, Australia.
¹⁹ Independent lay member, unaffiliated, Cheshire, UK.
²⁰ Department of Obstetrics and Gynaecology, Royal Women's Hospital, University of Melbourne, Melbourne, Australia.
²¹ Department of Ophthalmology, University of Colorado Anschutz Medical Campus.
²² Department of Obstetrics and Gynaecology, Monash University, VIC, Australia.
²³ The BMJ, London, UK.
²⁴ Department of Neurology, Brigham and Women's Hospital, Boston, MA, USA.
²⁵ Cochrane Denmark & Centre for Evidence-Based Medicine Odense, Department of Clinical Research, University of Southern Denmark, Denmark.
²⁶ Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital, Bispebjerg and Frederiksberg, Denmark.
²⁷ School of Health and Society, University of Wollongong, Wollongong, Australia.
²⁸ Monash University, Clayton, Victoria, Australia.
²⁹ Aberdeen Centre for Women's Health Research, University of Aberdeen, Aberdeen, UK.
³⁰ Centre for Health and Wellbeing across the Lifecourse, Dept of Health Sciences, Brunel University London, UK.
³¹ School of Pharmacy, Charles Perkins Centre, The University of Sydney, Australia.
³² Division of Medical Ethics, New York University Grossman School of Medicine, New York, NY.
³³ Faculty of Medicine, University of New South Wales, Australia.
³⁴ Cochrane Central Production Service, UK.
³⁵ York Trials Unit, Dept of Health Sciences, University of York, UK.
³⁶ Cochrane Gynaecology and Fertility Satellite and Cochrane Sexually Transmitted Infection Group, Amsterdam.
³⁷ Reproduction and Development Research Institute, Amsterdam University Medical Center, The Netherlands.
³⁸ University of Colorado Anschutz Medical Campus, Colorado, USA.

PMID: 37873409
PMCID: PMC10593010
DOI: 10.1101/2023.09.21.23295626

Update in

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions.
Wilkinson J, Heal C, Antoniou GA, Flemyng E, Alfirevic Z, Avenell A, Barbour G, Brown NJL, Carlisle J, Clarke M, Dicker P, Dumville JC, Grey A, Grohmann S, Gurrin L, Hayden JA, Heathers J, Hunter KE, Lasserson T, Lam E, Lensen S, Li T, Li W, Loder E, Lundh A, Meyerowitz-Katz G, Mol BW, O'Connell NE, Parker L, Redman BK, Seidler L, Sheldrick KA, Sydenham E, Torgerson D, van Wely M, Wang R, Bero L, Kirkham JJ. Wilkinson J, et al. BMJ Open. 2024 Mar 11;14(3):e084164. doi: 10.1136/bmjopen-2024-084164. BMJ Open. 2024. PMID: 38471680 Free PMC article.

Abstract

Introduction: Randomised controlled trials (RCTs) inform healthcare decisions. It is now apparent that some published RCTs contain false data and some appear to have been entirely fabricated. Systematic reviews are performed to identify and synthesise all RCTs that have been conducted on a given topic. While it is usual to assess methodological features of the RCTs in the process of undertaking a systematic review, it is not usual to consider whether the RCTs contain false data. Studies containing false data therefore go unnoticed and contribute to systematic review conclusions. The INSPECT-SR project will develop a tool to assess the trustworthiness of RCTs in systematic reviews of healthcare related interventions.

Methods and analysis: The INSPECT-SR tool will be developed using expert consensus in combination with empirical evidence, over five stages: 1) a survey of experts to assemble a comprehensive list of checks for detecting problematic RCTs, 2) an evaluation of the feasibility and impact of applying the checks to systematic reviews, 3) a Delphi survey to determine which of the checks are supported by expert consensus, culminating in 4) a consensus meeting to select checks to be included in a draft tool and to determine its format, 5) prospective testing of the draft tool in the production of new health systematic reviews, to allow refinement based on user feedback. We anticipate that the INSPECT-SR tool will help researchers to identify problematic studies, and will help patients by protecting them from the influence of false data on their healthcare.

PubMed Disclaimer

Conflict of interest statement

Declaration of interests JW, CH, GA, LB, JJK declare funding from NIHR (NIHR203568) in relation to this work. LB additionally declares The University of Colorado receives remuneration for service as Senior Research Integrity Editor, Cochrane. WL, ALS, and RW declare funding from Australian National Health and Medical Research Council Investigator Grants (GNT2016729, GNT2009432, GNT2009767). EF, SG, TLa declare employment by Cochrane. Tla additionally declares authorship of a chapter in the Cochrane Handbook for Systematic Reviews of Interventions and that he is a developer of standards for Cochrane intervention reviews (MECIR). ES declares that she was a member of the Cochrane scientific misconduct policy advisory group. ZA declares he is a member of the Cochrane Library Editorial Board, and PI on a grant from Children Investment Foundation Fund to University of Liverpool to investigate research integrity of clinical trials related to nutritional supplements in pregnancy. TLi is supported by grant UG1 EY020522 from the National Eye Institute, National Institutes of Health. MC declares that he is Co-ordinating Editor for the Cochrane Methodology Review Group. AA declares that The Health Services Research Unit, University of Aberdeen, is funded by the Health and Social Care Directorates of the Scottish Government.

Figures

**Figure 1:**
INSPECT-SR development process.

See this image and copyright information in PMC

References

1. Bryant A, Lawrie TA, Fordham EJ. Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines. American Journal of Therapeutics, 28, e434-e460, July 2021. Am J Ther. 2021;28(5):e573–e6. - PMC - PubMed
1. Hill A, Garratt A, Levi J, Falconer J, Ellis L, McCann K, et al. Meta-analysis of Randomized Trials of Ivermectin to Treat SARS-CoV-2 Infection. Open Forum Infect Dis. 2021;8(11):ofab358. - PMC - PubMed
1. Lawrence JM, Meyerowitz-Katz G, Heathers JAJ, Brown NJL, Sheldrick KA. The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable. Nat Med. 2021;27(11):1853–4. - PubMed
1. Brown NJL. http://steamtraen.blogspot.com/2021. [cited 2023]. Available from: http://steamtraen.blogspot.com/2021/07/Some-problems-with-the-data-from-....
1. Hill A, Mirchandani M, Pilkington V. Ivermectin for COVID-19: Addressing Potential Bias and Medical Fraud. Open Forum Infect Dis. 2022;9(2):ofab645. - PMC - PubMed

Publication types

Actions

Grants and funding

UG1 EY020522/EY/NEI NIH HHS/United States

LinkOut - more resources

Full Text Sources
Research Materials
- NCI CPTC Antibody Characterization Program

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

This is a preprint.

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions

Affiliations

Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions

Authors

Affiliations

Update in

Abstract

Conflict of interest statement

Figures

References

Publication types

Grants and funding

LinkOut - more resources

Full Text Sources

Research Materials