Automated quantitative MRI volumetry reports support diagnostic interpretation in dementia: a multi-rater, clinical accuracy study

Hugh G Pemberton^#^{1

2

3}, Olivia Goodkin^#^{4

5}, Ferran Prados^{4

6}, Ravi K Das⁷, Sjoerd B Vos^{4

5}, James Moggridge^{5

8}, William Coath⁹, Elizabeth Gordon⁹, Ryan Barrett¹⁰, Anne Schmitt^{5

8}, Hefina Whiteley-Jones¹⁰, Christian Burd¹¹, Mike P Wattjes¹², Sven Haller¹³, Meike W Vernooij^{14

15}, Lorna Harper⁹, Nick C Fox⁹, Ross W Paterson⁹, Jonathan M Schott⁹, Sotirios Bisdas^{5

8}, Mark White¹⁶, Sebastien Ourselin¹⁷, John S Thornton^{5

8}, Tarek A Yousry^{5

8}, M Jorge Cardoso¹⁷, Frederik Barkhof^{4

5

8

18}; Alzheimer’s Disease Neuroimaging Initiative

Affiliations

¹ Centre for Medical Image Computing (CMIC), Department of Medical Physics and Bioengineering, University College London, London, UK. h.pemberton@ucl.ac.uk.
² Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK. h.pemberton@ucl.ac.uk.
³ Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK. h.pemberton@ucl.ac.uk.
⁴ Centre for Medical Image Computing (CMIC), Department of Medical Physics and Bioengineering, University College London, London, UK.
⁵ Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK.
⁶ Universitat Oberta de Catalunya, Barcelona, Spain.
⁷ Clinical, Educational and Health Psychology, University College London, London, UK.
⁸ Lysholm Department of Neuroradiology, National Hospital for Neurology and Neurosurgery, UCLH NHS Foundation Trust, London, UK.
⁹ Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK.
¹⁰ Department of Neuroradiology, Brighton and Sussex University Hospitals, Brighton, UK.
¹¹ Guy's and St Thomas' NHS Foundation Trust, London, UK.
¹² Department of Diagnostic and Interventional Neuroradiology, Hannover Medical School, Hannover, Germany.
¹³ Department of Surgical Sciences, Radiology, Uppsala University, Uppsala, Sweden.
¹⁴ Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center, Rotterdam, the Netherlands.
¹⁵ Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, the Netherlands.
¹⁶ Digital Services, University College London Hospital, London, UK.
¹⁷ School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
¹⁸ Radiology & Nuclear Medicine, VU University Medical Center, Amsterdam, the Netherlands.

^# Contributed equally.

PMID: 33452627
PMCID: PMC8213665
DOI: 10.1007/s00330-020-07455-8

Automated quantitative MRI volumetry reports support diagnostic interpretation in dementia: a multi-rater, clinical accuracy study

Hugh G Pemberton et al. Eur Radiol. 2021 Jul.

. 2021 Jul;31(7):5312-5323.

doi: 10.1007/s00330-020-07455-8. Epub 2021 Jan 15.

Authors

Affiliations

¹ Centre for Medical Image Computing (CMIC), Department of Medical Physics and Bioengineering, University College London, London, UK. h.pemberton@ucl.ac.uk.
² Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK. h.pemberton@ucl.ac.uk.
³ Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK. h.pemberton@ucl.ac.uk.
⁴ Centre for Medical Image Computing (CMIC), Department of Medical Physics and Bioengineering, University College London, London, UK.
⁵ Neuroradiological Academic Unit, UCL Queen Square Institute of Neurology, University College London, London, UK.
⁶ Universitat Oberta de Catalunya, Barcelona, Spain.
⁷ Clinical, Educational and Health Psychology, University College London, London, UK.
⁸ Lysholm Department of Neuroradiology, National Hospital for Neurology and Neurosurgery, UCLH NHS Foundation Trust, London, UK.
⁹ Dementia Research Centre, UCL Queen Square Institute of Neurology, University College London, London, UK.
¹⁰ Department of Neuroradiology, Brighton and Sussex University Hospitals, Brighton, UK.
¹¹ Guy's and St Thomas' NHS Foundation Trust, London, UK.
¹² Department of Diagnostic and Interventional Neuroradiology, Hannover Medical School, Hannover, Germany.
¹³ Department of Surgical Sciences, Radiology, Uppsala University, Uppsala, Sweden.
¹⁴ Department of Radiology and Nuclear Medicine, Erasmus MC University Medical Center, Rotterdam, the Netherlands.
¹⁵ Department of Epidemiology, Erasmus MC University Medical Center, Rotterdam, the Netherlands.
¹⁶ Digital Services, University College London Hospital, London, UK.
¹⁷ School of Biomedical Engineering and Imaging Sciences, King's College London, London, UK.
¹⁸ Radiology & Nuclear Medicine, VU University Medical Center, Amsterdam, the Netherlands.

^# Contributed equally.

PMID: 33452627
PMCID: PMC8213665
DOI: 10.1007/s00330-020-07455-8

Abstract

Objectives: We examined whether providing a quantitative report (QReport) of regional brain volumes improves radiologists' accuracy and confidence in detecting volume loss, and in differentiating Alzheimer's disease (AD) and frontotemporal dementia (FTD), compared with visual assessment alone.

Methods: Our forced-choice multi-rater clinical accuracy study used MRI from 16 AD patients, 14 FTD patients, and 15 healthy controls; age range 52-81. Our QReport was presented to raters with regional grey matter volumes plotted as percentiles against data from a normative population (n = 461). Nine raters with varying radiological experience (3 each: consultants, registrars, 'non-clinical image analysts') assessed each case twice (with and without the QReport). Raters were blinded to clinical and demographic information; they classified scans as 'normal' or 'abnormal' and if 'abnormal' as 'AD' or 'FTD'.

Results: The QReport improved sensitivity for detecting volume loss and AD across all raters combined (p = 0.015* and p = 0.002*, respectively). Only the consultant group's accuracy increased significantly when using the QReport (p = 0.02*). Overall, raters' agreement (Cohen's κ) with the 'gold standard' was not significantly affected by the QReport; only the consultant group improved significantly (κ_s 0.41➔0.55, p = 0.04*). Cronbach's alpha for interrater agreement improved from 0.886 to 0.925, corresponding to an improvement from 'good' to 'excellent'.

Conclusion: Our QReport referencing single-subject results to normative data alongside visual assessment improved sensitivity, accuracy, and interrater agreement for detecting volume loss. The QReport was most effective in the consultants, suggesting that experience is needed to fully benefit from the additional information provided by quantitative analyses.

Key points: • The use of quantitative report alongside routine visual MRI assessment improves sensitivity and accuracy for detecting volume loss and AD vs visual assessment alone. • Consultant neuroradiologists' assessment accuracy and agreement (kappa scores) significantly improved with the use of quantitative atrophy reports. • First multi-rater radiological clinical evaluation of visual quantitative MRI atrophy report for use as a diagnostic aid in dementia.

Keywords: Alzheimer’s disease; Atrophy; Frontotemporal dementia; Magnetic resonance imaging; Radiologists.

PubMed Disclaimer

Conflict of interest statement

SH is a speaker for General Electric, a consultant for Spineart and on the imaging advisory board for the European Prevention of Alzheimer’s Disease. JC and SO are founders of Brainminer. MPW received speaker and/or consultancy fees from Bayer, Biogen, Biologix, Celgene, GeniLac, Imcyse, Medison, Merck-Serono, Novartis, Sanofi Genzyme, and Roche. FB is a board member for Neurology, Brain, Radiology, and MSJ; section editor for Neuroradiology; personal fees from Bayer, Biogen, Roche, IXICO, and GeNeuro; grants from Novartis, Teva Pharmaceuticals, Merck, Biogen, Innovative Medicines Initiative, General Electric Healthcare, UK MS Society, Dutch Foundation MS Research, NWO, and NIHR. JMS has received research funding and PET tracer from AVID Radiopharmaceuticals (a wholly owned subsidiary of Eli Lilly); has consulted for Roche, Eli Lilly, Biogen, Merck, and GE; received royalties from Oxford University Press and Henry Stewart Talks; given education lectures sponsored by Eli Lilly, Biogen, and GE; and served on a Data Safety Monitoring Committee for Axon Neuroscience SE. He is Chief Medical Officer for Alzheimer’s Research UK.

Figures

**Fig. 1**
Quantitative report (QReport) of an AD patient displaying demographics, hippocampal volume percentiles, and single-subject brain parenchymal fraction (red dot) plotted against a normative dementia-free population. Quality control metrics and a ‘rose plot’ representation of GM volume percentiles split by brain lobe and relevant sub-regions. The rose plot is on a log scale and uses a traffic light colour-coding system (green to red meaning high to low percentile) to display the individual’s volume percentiles in the context of a healthy population. *Abbreviations*: BPF, brain parenchymal fraction; SNR, signal-to-noise ratio; CNR, contrast-to-noise ratio; GM, grey matter; WM, white matter; CAU, caudate

**Fig. 2**
Screenshot from the Quantitative Neuroradiology Initiative (QNI) study website (http://qni.cs.ucl.ac.uk) showing the image viewer for a case with the QReport available. QReports were fully interactive and zoomable via the website

See this image and copyright information in PMC

References

1. Dubois B, Feldman HH, Jacova C, et al. Research criteria for the diagnosis of Alzheimer’s disease: revising the NINCDS–ADRDA criteria. Lancet Neurol. 2007;6:734–746. doi: 10.1016/S1474-4422(07)70178-3. - DOI - PubMed
1. Jack CR, Bennett DA, Blennow K, et al. NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimers Dement. 2018;14:535–562. doi: 10.1016/j.jalz.2018.02.018. - DOI - PMC - PubMed
1. Duchesne S, Caroli A, Geroldi C, Barillot C, Frisoni GB, Collins DL (2008) MRI-based automated computer classification of probable AD versus normal controls. IEEE Trans Med Imaging 27:509–520. 10.1109/TMI.2007.908685 - PubMed
1. ten Kate M, Barkhof F, Boccardi M et al (2017) Clinical validity of medial temporal atrophy as a biomarker for Alzheimer’s disease in the context of a structured 5-phase development framework. Neurobiol Aging. 10.1016/j.neurobiolaging.2016.05.024 - PubMed
1. Winblad B, Amouyel P, Andrieu S, et al. Defeating Alzheimer’s disease and other dementias: a priority for European science and society. Lancet Neurol. 2016;15:455–532. doi: 10.1016/S1474-4422(16)00062-4. - DOI - PubMed

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

LinkOut - more resources

Full Text Sources
Other Literature Sources
- scite Smart Citations
Medical
- MedlinePlus Health Information

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

Automated quantitative MRI volumetry reports support diagnostic interpretation in dementia: a multi-rater, clinical accuracy study

Affiliations

Automated quantitative MRI volumetry reports support diagnostic interpretation in dementia: a multi-rater, clinical accuracy study

Authors

Affiliations

Abstract

Conflict of interest statement

Figures

References

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Medical