Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Samantha Cruz Rivera^{1

2

3}, Xiaoxuan Liu^{3

4

5

6

7}, An-Wen Chan⁸, Alastair K Denniston^{9

10

11

12

13

14}, Melanie J Calvert^{1

2

3

6

15

16

17}; SPIRIT-AI and CONSORT-AI Working Group; SPIRIT-AI and CONSORT-AI Steering Group; SPIRIT-AI and CONSORT-AI Consensus Group

Collaborators, Affiliations

Collaborators

Affiliations

¹ Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
² Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
³ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
⁴ Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
⁵ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
⁶ Health Data Research UK, London, UK.
⁷ Moorfields Eye Hospital NHS Foundation Trust, London, UK.
⁸ Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Ontario, Canada.
⁹ Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹⁰ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹¹ Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹² University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. a.denniston@bham.ac.uk.
¹³ Health Data Research UK, London, UK. a.denniston@bham.ac.uk.
¹⁴ National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK. a.denniston@bham.ac.uk.
¹⁵ National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹⁶ National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, UK.
¹⁷ National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.

PMID: 32908284
PMCID: PMC7598944
DOI: 10.1038/s41591-020-1037-7

Review

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Samantha Cruz Rivera et al. Nat Med. 2020 Sep.

. 2020 Sep;26(9):1351-1363.

doi: 10.1038/s41591-020-1037-7. Epub 2020 Sep 9.

Authors

Collaborators

Affiliations

¹ Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
² Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
³ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
⁴ Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK.
⁵ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
⁶ Health Data Research UK, London, UK.
⁷ Moorfields Eye Hospital NHS Foundation Trust, London, UK.
⁸ Department of Medicine, Women's College Research Institute, Women's College Hospital, University of Toronto, Ontario, Canada.
⁹ Centre for Patient Reported Outcomes Research, Institute of Applied Health Research, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹⁰ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹¹ Academic Unit of Ophthalmology, Institute of Inflammation and Ageing, University of Birmingham, Birmingham, UK. a.denniston@bham.ac.uk.
¹² University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK. a.denniston@bham.ac.uk.
¹³ Health Data Research UK, London, UK. a.denniston@bham.ac.uk.
¹⁴ National Institute of Health Research Biomedical Research Centre for Ophthalmology, Moorfields Hospital London NHS Foundation Trust and University College London, Institute of Ophthalmology, London, UK. a.denniston@bham.ac.uk.
¹⁵ National Institute of Health Research Birmingham Biomedical Research Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹⁶ National Institute of Health Research Applied Research Collaborative West Midlands, Coventry, UK.
¹⁷ National Institute of Health Research Surgical Reconstruction and Microbiology Centre, University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.

PMID: 32908284
PMCID: PMC7598944
DOI: 10.1038/s41591-020-1037-7

Abstract

The SPIRIT 2013 statement aims to improve the completeness of clinical trial protocol reporting by providing evidence-based recommendations for the minimum set of items to be addressed. This guidance has been instrumental in promoting transparent evaluation of new interventions. More recently, there has been a growing recognition that interventions involving artificial intelligence (AI) need to undergo rigorous, prospective evaluation to demonstrate their impact on health outcomes. The SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) extension is a new reporting guideline for clinical trial protocols evaluating interventions with an AI component. It was developed in parallel with its companion statement for trial reports: CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence). Both guidelines were developed through a staged consensus process involving literature review and expert consultation to generate 26 candidate items, which were consulted upon by an international multi-stakeholder group in a two-stage Delphi survey (103 stakeholders), agreed upon in a consensus meeting (31 stakeholders) and refined through a checklist pilot (34 participants). The SPIRIT-AI extension includes 15 new items that were considered sufficiently important for clinical trial protocols of AI interventions. These new items should be routinely reported in addition to the core SPIRIT 2013 items. SPIRIT-AI recommends that investigators provide clear descriptions of the AI intervention, including instructions and skills required for use, the setting in which the AI intervention will be integrated, considerations for the handling of input and output data, the human-AI interaction and analysis of error cases. SPIRIT-AI will help promote transparency and completeness for clinical trial protocols for AI interventions. Its use will assist editors and peer reviewers, as well as the general readership, to understand, interpret and critically appraise the design and risk of bias for a planned clinical trial.

PubMed Disclaimer

Conflict of interest statement

M.J.C. has received personal fees from Astellas, Takeda, Merck, Daiichi Sankyo, Glaukos, GlaxoSmithKline and the Patient-Centered Outcomes Research Institute (PCORI) outside the submitted work. P.A.K. is a consultant for DeepMind Technologies, Roche, Novartis and Apellis, and has received speaker fees or travel support from Bayer, Allergan, Topcon and Heidelberg Engineering. C.J.K. is an employee of Google and owns Alphabet stock. A.E. is an employee of Salesforce CRM. R.S. is an employee of Pinpoint Science. J. Matcham was an employee of AstraZeneca at the time of this study. J. Monteiro is Chief Editor of the journal Nature Medicine; he has recused himself from any aspect of decision-making on this manuscript and played no part in the assignment of this manuscript to in-house editors or peer reviewers, and was also separated and blinded from the editorial process from submission inception to decision.

Figures

**Fig. 1. CONSORT 2010 flow diagram — adapted for AI clinical trials.**
SPIRIT-AI 10 (i): State the inclusion and exclusion criteria at the level of participants. SPIRIT-AI 10 (ii): State the inclusion and exclusion criteria at the level of the input data. SPIRIT 13 (core CONSORT item): Time schedule of enrollment, interventions (including any run-ins and washouts), assessments, and visits for participants. A schematic diagram is highly recommended.

See this image and copyright information in PMC

References

1. Chan A-W, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann. Intern. Med. 2013;158:200–207. - PMC - PubMed
1. Chan A-W, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. Br. Med. J. 2013;346:e7586. - PMC - PubMed
1. Sarkis-Onofre R, et al. Use of guidelines to improve the quality and transparency of reporting oral health research. J. Dent. 2015;43:397–404. - PubMed
1. Calvert M, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: The SPIRIT-PRO Extension. J. Am. Med. Assoc. 2018;319:483–494. - PubMed
1. Dai L, et al. Standard protocol items for clinical trials with traditional Chinese medicine 2018: recommendations, explanation and elaboration (SPIRIT-TCM Extension 2018). Chin. J. Integr. Med. 2019;25:71–79. - PubMed

Publication types

Actions
Actions
Actions
Actions
Actions

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Collaborators

Affiliations

Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension

Authors

Collaborators

Affiliations

Abstract

Conflict of interest statement

Figures

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Medical