A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis

Michael Barnett^#^{1

2

3}, Dongang Wang^#^{1

2}, Heidi Beadnall^{2

3}, Antje Bischof⁴, David Brunacci⁵, Helmut Butzkueven^{6

7}, J William L Brown⁸, Mariano Cabezas², Tilak Das⁹, Tej Dugal^{1

10}, Daniel Guilfoyle³, Alexander Klistorner^{1

2

11}, Stephen Krieger¹², Kain Kyle^{1

2}, Linda Ly¹, Lynette Masters¹³, Andy Shieh¹, Zihao Tang^{1

2}, Anneke van der Walt^{5

6}, Kayla Ward³, Heinz Wiendl⁴, Geng Zhan^{1

2}, Robert Zivadinov¹⁴, Yael Barnett^{1

15}, Chenyu Wang^{16

17}

Affiliations

¹ Sydney Neuroimaging Analysis Centre, Sydney, NSW, Australia.
² Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia.
³ Department of Neurology, Royal Prince Alfred Hospital, Camperdown, NSW, Australia.
⁴ Department of Neurology, University Hospital of Muenster, Muenster, Germany.
⁵ Department of Radiology, Royal Prince Alfred Hospital, Camperdown, NSW, Australia.
⁶ Department of Neurology, The Alfred Hospital, Melbourne, VIC, Australia.
⁷ Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia.
⁸ Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK.
⁹ Department of Radiology, University of Cambridge, Cambridge, UK.
¹⁰ Synergy Radiology, Sydney, NSW, Australia.
¹¹ Save Sight Institute, University of Sydney, Sydney, NSW, Australia.
¹² Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
¹³ I-MED Radiology, Sydney, NSW, Australia.
¹⁴ Buffalo Neuroimaging Analysis Centre, Buffalo, NY, USA.
¹⁵ Department of Radiology, St Vincent's Hospital, Sydney, NSW, Australia.
¹⁶ Sydney Neuroimaging Analysis Centre, Sydney, NSW, Australia. tim@snac.com.au.
¹⁷ Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia. tim@snac.com.au.

^# Contributed equally.

PMID: 37857813
PMCID: PMC10587188
DOI: 10.1038/s41746-023-00940-6

A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis

Michael Barnett et al. NPJ Digit Med. 2023.

. 2023 Oct 19;6(1):196.

doi: 10.1038/s41746-023-00940-6.

Authors

Affiliations

¹ Sydney Neuroimaging Analysis Centre, Sydney, NSW, Australia.
² Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia.
³ Department of Neurology, Royal Prince Alfred Hospital, Camperdown, NSW, Australia.
⁴ Department of Neurology, University Hospital of Muenster, Muenster, Germany.
⁵ Department of Radiology, Royal Prince Alfred Hospital, Camperdown, NSW, Australia.
⁶ Department of Neurology, The Alfred Hospital, Melbourne, VIC, Australia.
⁷ Department of Neuroscience, Central Clinical School, Monash University, Melbourne, VIC, Australia.
⁸ Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK.
⁹ Department of Radiology, University of Cambridge, Cambridge, UK.
¹⁰ Synergy Radiology, Sydney, NSW, Australia.
¹¹ Save Sight Institute, University of Sydney, Sydney, NSW, Australia.
¹² Department of Neurology, Icahn School of Medicine at Mount Sinai, New York, NY, USA.
¹³ I-MED Radiology, Sydney, NSW, Australia.
¹⁴ Buffalo Neuroimaging Analysis Centre, Buffalo, NY, USA.
¹⁵ Department of Radiology, St Vincent's Hospital, Sydney, NSW, Australia.
¹⁶ Sydney Neuroimaging Analysis Centre, Sydney, NSW, Australia. tim@snac.com.au.
¹⁷ Brain and Mind Centre, University of Sydney, Sydney, NSW, Australia. tim@snac.com.au.

^# Contributed equally.

PMID: 37857813
PMCID: PMC10587188
DOI: 10.1038/s41746-023-00940-6

Abstract

Modern management of MS targets No Evidence of Disease Activity (NEDA): no clinical relapses, no magnetic resonance imaging (MRI) disease activity and no disability worsening. While MRI is the principal tool available to neurologists for monitoring clinically silent MS disease activity and, where appropriate, escalating treatment, standard radiology reports are qualitative and may be insensitive to the development of new or enlarging lesions. Existing quantitative neuroimaging tools lack adequate clinical validation. In 397 multi-center MRI scan pairs acquired in routine practice, we demonstrate superior case-level sensitivity of a clinically integrated AI-based tool over standard radiology reports (93.3% vs 58.3%), relative to a consensus ground truth, with minimal loss of specificity. We also demonstrate equivalence of the AI-tool with a core clinical trial imaging lab for lesion activity and quantitative brain volumetric measures, including percentage brain volume loss (PBVC), an accepted biomarker of neurodegeneration in MS (mean PBVC -0.32% vs -0.36%, respectively), whereas even severe atrophy (>0.8% loss) was not appreciated in radiology reports. Finally, the AI-tool additionally embeds a clinically meaningful, experiential comparator that returns a relevant MS patient centile for lesion burden, revealing, in our cohort, inconsistencies in qualitative descriptors used in radiology reports. AI-based image quantitation enhances the accuracy of, and value-adds to, qualitative radiology reporting. Scaled deployment of these tools will open a path to precision management for patients with MS.

PubMed Disclaimer

Conflict of interest statement

The authors declare the following financial/non-financial competing interests: D.W., K.K., L.L., A.S., Z.T., G.Z. and C.W. are employees of Sydney Neuroimaging Analysis Centre (SNAC); M.B. is a consulting neurologist and director for research at SNAC; T.D. and Y.B. are consulting radiologists at SNAC. C.W. and Y.B. personally hold equity ownership in SNAC, the entity that manufactures iQ-Solutions^TM

Figures

**Fig. 1. iQ-Solutions^TM PACS integration.**
iQ-MS automatically returns a co-registered baseline (prior study) 3D FLAIR series together with a lesion-annotated 3D FLAIR, here showing a case with both new (blue) and enlarging (green) lesions. A 3D-T1 series is also returned with both whole brain (yellow) and thalamus (pink) annotations. From left to right images : Co-registered FLAIR from patient’s last scan; FLAIR image of the current MRI exam; Lesion masks overlaid on current FLAIR image; Brain and Thalamus masks overlaid on current 3DT1 image.

**Fig. 2. Correlation of annualized percentage brain volume change between scan pairs as determined by SIENA and iQ-MS.**
PBVC percentage brain volume change.

See this image and copyright information in PMC

References

1. Walton C, et al. Rising prevalence of multiple sclerosis worldwide: Insights from the Atlas of MS, third edition. Mult. Scler. 2020;26:1816–1821. doi: 10.1177/1352458520970841. - DOI - PMC - PubMed
1. Cree BAC, et al. Silent progression in disease activity-free relapsing multiple sclerosis. Ann. Neurol. 2019;85:653–666. doi: 10.1002/ana.25463. - DOI - PMC - PubMed
1. Kappos L, et al. Contribution of relapse-independent progression vs relapse-associated worsening to overall confirmed disability accumulation in typical relapsing multiple sclerosis in a pooled analysis of 2 randomized clinical trials. JAMA Neurol. 2020;77:1132–1140. doi: 10.1001/jamaneurol.2020.1568. - DOI - PMC - PubMed
1. Sharrad D, Chugh P, Slee M, Bacchi S. Defining progression independent of relapse activity (PIRA) in adult patients with relapsing multiple sclerosis: a systematic review. Mult. Scler. Relat. Disord. 2023;78:104899. doi: 10.1016/j.msard.2023.104899. - DOI - PubMed
1. Barnett M, Barnett Y, Reddel S. MRI and laboratory monitoring of disease-modifying therapy efficacy and risks. Curr. Opin. Neurol. 2022;35:278–285. doi: 10.1097/WCO.0000000000001067. - DOI - PubMed

Grants and funding

LinkOut - more resources

Full Text Sources
Miscellaneous
- NCI CPTAC Assay Portal

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis

Affiliations

A real-world clinical validation for AI-based MRI monitoring in multiple sclerosis

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

Grants and funding

LinkOut - more resources

Full Text Sources

Miscellaneous