A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Viknesh Sounderajah^{1

2}, Hutan Ashrafian^{3

4}, Sherri Rose⁵, Nigam H Shah⁶, Marzyeh Ghassemi⁷, Robert Golub⁸, Charles E Kahn Jr⁹, Andre Esteva¹⁰, Alan Karthikesalingam¹¹, Bilal Mateen¹², Dale Webster¹¹, Dan Milea¹³, Daniel Ting¹³, Darren Treanor^{14

15

16

17}, Dominic Cushnan¹⁸, Dominic King^{3

19}, Duncan McPherson²⁰, Ben Glocker²¹, Felix Greaves²², Leanne Harling^{3

4}, Johan Ordish²⁰, Jérémie F Cohen²³, Jon Deeks²⁴, Mariska Leeflang²⁵, Matthew Diamond²⁶, Matthew D F McInnes²⁷, Melissa McCradden²⁸, Michael D Abràmoff²⁹, Pasha Normahani⁴, Sheraz R Markar⁴, Stephanie Chang³⁰, Xiaoxuan Liu^{31

32

33}, Susan Mallett³⁴, Shravya Shetty¹¹, Alastair Denniston^{31

32

33}, Gary S Collins^{35

36}, David Moher³⁷, Penny Whiting³⁸, Patrick M Bossuyt³⁹, Ara Darzi^{40

41}

Affiliations

¹ Institute of Global Health Innovation, Imperial College London, London, UK. viknesh.sounderajah08@imperial.ac.uk.
² Department of Surgery and Cancer, Imperial College London, London, UK. viknesh.sounderajah08@imperial.ac.uk.
³ Institute of Global Health Innovation, Imperial College London, London, UK.
⁴ Department of Surgery and Cancer, Imperial College London, London, UK.
⁵ Center for Health Policy and Center for Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA.
⁶ Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA.
⁷ Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
⁸ Journal of the American Medical Association (JAMA), Chicago, IL, USA.
⁹ University of Pennsylvania, Philadelphia, Pennsylvania, PA, USA.
¹⁰ Salesforce Research, San Francisco, CA, USA.
¹¹ Google Health, Palo Alto, CA, USA.
¹² Wellcome Trust, London, UK.
¹³ Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.
¹⁴ Leeds Teaching Hospitals NHS Trust, Leeds, UK.
¹⁵ University of Leeds, Leeds, UK.
¹⁶ Department of Clinical Pathology, and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.
¹⁷ Center for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden.
¹⁸ NHSX, London, UK.
¹⁹ Optum, London, UK.
²⁰ Medicines and Healthcare Products Regulatory Agency, London, UK.
²¹ Faculty of Engineering, Department of Computing, Imperial College London, London, UK.
²² National Institute for Health and Care Excellence, London, UK.
²³ Department of Pediatrics, Centre of Research in Epidemiology and Statistics, Inserm UMR 1153, Necker- Enfants Malades Hospital, Assistance Publique-Hôpitaux de Paris, Université de Paris, Paris, France.
²⁴ Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
²⁵ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands.
²⁶ Food and Drug Administration, Silver Spring, MD, USA.
²⁷ Departments of Radiology and Epidemiology, University of Ottawa, The Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
²⁸ Department of Bioethics, The Hospital for Sick Kids, Toronto, Ontario, Canada.
²⁹ Department of Ophthalmology and Visual Sciences, University of Iowa, Iowa City, IA, USA.
³⁰ Annals of Internal Medicine, American College of Physicians, Philadelphia, PA, USA.
³¹ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
³² University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
³³ Health Data Research UK, London, UK.
³⁴ Centre for Medical Imaging, University College London, London, UK.
³⁵ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
³⁶ NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
³⁷ Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
³⁸ Bristol Medical School, University of Bristol, Bristol, UK.
³⁹ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands. p.m.bossuyt@amsterdamumc.nl.
⁴⁰ Institute of Global Health Innovation, Imperial College London, London, UK. a.darzi@imperial.ac.uk.
⁴¹ Department of Surgery and Cancer, Imperial College London, London, UK. a.darzi@imperial.ac.uk.

PMID: 34635854
DOI: 10.1038/s41591-021-01517-0

A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Viknesh Sounderajah et al. Nat Med. 2021 Oct.

. 2021 Oct;27(10):1663-1665.

doi: 10.1038/s41591-021-01517-0.

Authors

Affiliations

¹ Institute of Global Health Innovation, Imperial College London, London, UK. viknesh.sounderajah08@imperial.ac.uk.
² Department of Surgery and Cancer, Imperial College London, London, UK. viknesh.sounderajah08@imperial.ac.uk.
³ Institute of Global Health Innovation, Imperial College London, London, UK.
⁴ Department of Surgery and Cancer, Imperial College London, London, UK.
⁵ Center for Health Policy and Center for Primary Care and Outcomes Research, Stanford University, Stanford, CA, USA.
⁶ Center for Biomedical Informatics Research, Stanford University, Stanford, CA, USA.
⁷ Institute for Medical Engineering & Science, Massachusetts Institute of Technology, Cambridge, MA, USA.
⁸ Journal of the American Medical Association (JAMA), Chicago, IL, USA.
⁹ University of Pennsylvania, Philadelphia, Pennsylvania, PA, USA.
¹⁰ Salesforce Research, San Francisco, CA, USA.
¹¹ Google Health, Palo Alto, CA, USA.
¹² Wellcome Trust, London, UK.
¹³ Singapore Eye Research Institute, Singapore National Eye Centre, Singapore, Singapore.
¹⁴ Leeds Teaching Hospitals NHS Trust, Leeds, UK.
¹⁵ University of Leeds, Leeds, UK.
¹⁶ Department of Clinical Pathology, and Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden.
¹⁷ Center for Medical Image Science and Visualization (CMIV), Linköping University, Linköping, Sweden.
¹⁸ NHSX, London, UK.
¹⁹ Optum, London, UK.
²⁰ Medicines and Healthcare Products Regulatory Agency, London, UK.
²¹ Faculty of Engineering, Department of Computing, Imperial College London, London, UK.
²² National Institute for Health and Care Excellence, London, UK.
²³ Department of Pediatrics, Centre of Research in Epidemiology and Statistics, Inserm UMR 1153, Necker- Enfants Malades Hospital, Assistance Publique-Hôpitaux de Paris, Université de Paris, Paris, France.
²⁴ Institute of Applied Health Research, University of Birmingham, Birmingham, UK.
²⁵ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands.
²⁶ Food and Drug Administration, Silver Spring, MD, USA.
²⁷ Departments of Radiology and Epidemiology, University of Ottawa, The Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
²⁸ Department of Bioethics, The Hospital for Sick Kids, Toronto, Ontario, Canada.
²⁹ Department of Ophthalmology and Visual Sciences, University of Iowa, Iowa City, IA, USA.
³⁰ Annals of Internal Medicine, American College of Physicians, Philadelphia, PA, USA.
³¹ Institute of Inflammation and Ageing, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK.
³² University Hospitals Birmingham NHS Foundation Trust, Birmingham, UK.
³³ Health Data Research UK, London, UK.
³⁴ Centre for Medical Imaging, University College London, London, UK.
³⁵ Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK.
³⁶ NIHR Oxford Biomedical Research Centre, Oxford University Hospitals NHS Foundation Trust, Oxford, UK.
³⁷ Ottawa Hospital Research Institute, Ottawa, Ontario, Canada.
³⁸ Bristol Medical School, University of Bristol, Bristol, UK.
³⁹ Department of Epidemiology and Data Science, Amsterdam University Medical Centres, University of Amsterdam, Amsterdam, The Netherlands. p.m.bossuyt@amsterdamumc.nl.
⁴⁰ Institute of Global Health Innovation, Imperial College London, London, UK. a.darzi@imperial.ac.uk.
⁴¹ Department of Surgery and Cancer, Imperial College London, London, UK. a.darzi@imperial.ac.uk.

PMID: 34635854
DOI: 10.1038/s41591-021-01517-0

No abstract available

PubMed Disclaimer

References

1. Benjamens, S., Dhunnoo, P. & Meskó, B. npj Digit. Med. 3, 118 (2020). - DOI
1. Jayakumar, S. et al. npj Digital Med. (in the press).
1. Whiting, P. F. Ann. Intern. Med. 155, 529 (2011). - DOI
1. Page, M. J. et al. BMJ. 372, n71 (2021). - DOI
1. Yang, B. et al. Open Science Framework https://doi.org/10.17605/OSF.IO/HQ8MF (2018).

Publication types

Actions
Actions

MeSH terms

Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
- Nature Publishing Group
Medical
- MedlinePlus Health Information

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Affiliations

A quality assessment tool for artificial intelligence-centered diagnostic test accuracy studies: QUADAS-AI

Authors

Affiliations

References

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Medical