Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Viknesh Sounderajah^{1

2}, Hutan Ashrafian^{3

2}, Robert M Golub⁴, Shravya Shetty⁵, Jeffrey De Fauw⁶, Lotty Hooft^{7

8}, Karel Moons^{7

8}, Gary Collins⁹, David Moher¹⁰, Patrick M Bossuyt¹¹, Ara Darzi^{1

2}, Alan Karthikesalingam¹², Alastair K Denniston^{13

14

15

16}, Bilal Akhter Mateen¹⁷, Daniel Ting¹⁸, Darren Treanor¹⁹, Dominic King²⁰, Felix Greaves²¹, Jonathan Godwin⁶, Jonathan Pearson-Stuttard²², Leanne Harling¹, Matthew McInnes²³, Nader Rifai²⁴, Nenad Tomasev⁶, Pasha Normahani¹, Penny Whiting²⁵, Ravi Aggarwal^{1

2}, Sebastian Vollmer¹⁷, Sheraz R Markar¹, Trishan Panch²⁶, Xiaoxuan Liu^{13

14

15

16}; STARD-AI Steering Committee

Affiliations

¹ Department of Surgery and Cancer, Imperial College London, Paddington, UK.
² Institute of Global Health Innovation, Imperial College London, London, UK.
³ Department of Surgery and Cancer, Imperial College London, Paddington, UK hutan@imperial.ac.uk.
⁴ Journal of the American Medical Association, Chicago, Illinois, USA.
⁵ Google Health, Pal Alto, California, USA.
⁶ DeepMind Technologies Ltd, London, UK.
⁷ Cochrane Netherlands, University Medical Center Utrecht, University of Utrecht, Utrecht, The Netherlands.
⁸ Department of Epidemiology, Julius Center for Health Sciences and Primary Care, Utrecht, The Netherlands.
⁹ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
¹⁰ Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
¹¹ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, Duivendrecht, The Netherlands.
¹² Google Health, London, UK.
¹³ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
¹⁴ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹⁵ Health Data Research UK, London, UK.
¹⁶ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
¹⁷ The Alan Turing Institute, London, UK.
¹⁸ Singapore Eye Research Institute, Singapore National Eye Center, Singapore.
¹⁹ University of Leeds, Leeds, UK.
²⁰ Optum, Paddington, London, UK.
²¹ Department of Primary Care and Public Health, Imperial College London, London, UK.
²² School of Public Health, Imperial College London, London, UK.
²³ Department of Radiology, University of Ottawa, Ottawa, Ontario, Canada.
²⁴ Harvard Medical School, Boston, Massachusetts, USA.
²⁵ School of Social and Community Medicine, University of Bristol, Bristol, UK.
²⁶ Division of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.

PMID: 34183345
PMCID: PMC8240576
DOI: 10.1136/bmjopen-2020-047709

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Viknesh Sounderajah et al. BMJ Open. 2021.

. 2021 Jun 28;11(6):e047709.

doi: 10.1136/bmjopen-2020-047709.

Authors

Affiliations

¹ Department of Surgery and Cancer, Imperial College London, Paddington, UK.
² Institute of Global Health Innovation, Imperial College London, London, UK.
³ Department of Surgery and Cancer, Imperial College London, Paddington, UK hutan@imperial.ac.uk.
⁴ Journal of the American Medical Association, Chicago, Illinois, USA.
⁵ Google Health, Pal Alto, California, USA.
⁶ DeepMind Technologies Ltd, London, UK.
⁷ Cochrane Netherlands, University Medical Center Utrecht, University of Utrecht, Utrecht, The Netherlands.
⁸ Department of Epidemiology, Julius Center for Health Sciences and Primary Care, Utrecht, The Netherlands.
⁹ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
¹⁰ Centre for Journalology, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
¹¹ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, Duivendrecht, The Netherlands.
¹² Google Health, London, UK.
¹³ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
¹⁴ University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
¹⁵ Health Data Research UK, London, UK.
¹⁶ Birmingham Health Partners Centre for Regulatory Science and Innovation, University of Birmingham, Birmingham, UK.
¹⁷ The Alan Turing Institute, London, UK.
¹⁸ Singapore Eye Research Institute, Singapore National Eye Center, Singapore.
¹⁹ University of Leeds, Leeds, UK.
²⁰ Optum, Paddington, London, UK.
²¹ Department of Primary Care and Public Health, Imperial College London, London, UK.
²² School of Public Health, Imperial College London, London, UK.
²³ Department of Radiology, University of Ottawa, Ottawa, Ontario, Canada.
²⁴ Harvard Medical School, Boston, Massachusetts, USA.
²⁵ School of Social and Community Medicine, University of Bristol, Bristol, UK.
²⁶ Division of Health Policy and Management, Harvard T.H. Chan School of Public Health, Boston, Massachusetts, USA.

PMID: 34183345
PMCID: PMC8240576
DOI: 10.1136/bmjopen-2020-047709

Abstract

Introduction: Standards for Reporting of Diagnostic Accuracy Study (STARD) was developed to improve the completeness and transparency of reporting in studies investigating diagnostic test accuracy. However, its current form, STARD 2015 does not address the issues and challenges raised by artificial intelligence (AI)-centred interventions. As such, we propose an AI-specific version of the STARD checklist (STARD-AI), which focuses on the reporting of AI diagnostic test accuracy studies. This paper describes the methods that will be used to develop STARD-AI.

Methods and analysis: The development of the STARD-AI checklist can be distilled into six stages. (1) A project organisation phase has been undertaken, during which a Project Team and a Steering Committee were established; (2) An item generation process has been completed following a literature review, a patient and public involvement and engagement exercise and an online scoping survey of international experts; (3) A three-round modified Delphi consensus methodology is underway, which will culminate in a teleconference consensus meeting of experts; (4) Thereafter, the Project Team will draft the initial STARD-AI checklist and the accompanying documents; (5) A piloting phase among expert users will be undertaken to identify items which are either unclear or missing. This process, consisting of surveys and semistructured interviews, will contribute towards the explanation and elaboration document and (6) On finalisation of the manuscripts, the group's efforts turn towards an organised dissemination and implementation strategy to maximise end-user adoption.

Ethics and dissemination: Ethical approval has been granted by the Joint Research Compliance Office at Imperial College London (reference number: 19IC5679). A dissemination strategy will be aimed towards five groups of stakeholders: (1) academia, (2) policy, (3) guidelines and regulation, (4) industry and (5) public and non-specific stakeholders. We anticipate that dissemination will take place in Q3 of 2021.

Keywords: health informatics; protocols & guidelines; quality in health care.

PubMed Disclaimer

Conflict of interest statement

Competing interests: None declared.

References

1. Sounderajah V, Patel V, Varatharajan L, et al. . Are disruptive innovations recognised in the healthcare literature? A systematic review. BMJ Innov 2021;7:208–16. 10.1136/bmjinnov-2020-000424 - DOI - PMC - PubMed
1. Topol EJ. High-Performance medicine: the convergence of human and artificial intelligence. Nat Med 2019;25:44–56. 10.1038/s41591-018-0300-7 - DOI - PubMed
1. Benjamens S, Dhunnoo P, Meskó B. The state of artificial intelligence-based FDA-approved medical devices and algorithms: an online database. NPJ Digit Med 2020;3:118. 10.1038/s41746-020-00324-0 - DOI - PMC - PubMed
1. Williams BJ, Bottoms D, Treanor D. Future-proofing pathology: the case for clinical adoption of digital pathology. J Clin Pathol 2017;70:1010–8. 10.1136/jclinpath-2017-204644 - DOI - PubMed
1. Bossuyt PM, Reitsma JB, Bruns DE, et al. . STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies. BMJ 2015;351:h5527. 10.1136/bmj.h5527 - DOI - PMC - PubMed

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Medical
- ClinicalTrials.gov
- MedlinePlus Health Information
Research Materials
- NCI CPTC Antibody Characterization Program

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Affiliations

Developing a reporting guideline for artificial intelligence-centred diagnostic test accuracy studies: the STARD-AI protocol

Authors

Affiliations

Abstract

Conflict of interest statement

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Medical

Research Materials