VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images

Anjany Sekuboyina¹, Malek E Husseini², Amirhossein Bayat², Maximilian Löffler³, Hans Liebl³, Hongwei Li⁴, Giles Tetteh⁴, Jan Kukačka⁵, Christian Payer⁶, Darko Štern⁷, Martin Urschler⁸, Maodong Chen⁹, Dalong Cheng⁹, Nikolas Lessmann¹⁰, Yujin Hu¹¹, Tianfu Wang¹², Dong Yang¹³, Daguang Xu¹³, Felix Ambellan¹⁴, Tamaz Amiranashvili¹⁴, Moritz Ehlke¹⁵, Hans Lamecker¹⁵, Sebastian Lehnert¹⁵, Marilia Lirio¹⁵, Nicolás Pérez de Olaguer¹⁵, Heiko Ramm¹⁵, Manish Sahu¹⁴, Alexander Tack¹⁴, Stefan Zachow¹⁴, Tao Jiang¹⁶, Xinjun Ma¹⁶, Christoph Angerman¹⁷, Xin Wang¹⁸, Kevin Brown¹⁹, Alexandre Kirszenberg²⁰, Élodie Puybareau²⁰, Di Chen²¹, Yiwei Bai²¹, Brandon H Rapazzo²¹, Timyoas Yeah²², Amber Zhang²³, Shangliang Xu²⁴, Feng Hou²⁵, Zhiqiang He²⁶, Chan Zeng²⁷, Zheng Xiangshang²⁸, Xu Liming²⁹, Tucker J Netherton³⁰, Raymond P Mumme³⁰, Laurence E Court³⁰, Zixun Huang³¹, Chenhang He³², Li-Wen Wang³¹, Sai Ho Ling³³, Lê Duy Huỳnh²⁰, Nicolas Boutry²⁰, Roman Jakubicek³⁴, Jiri Chmelik³⁴, Supriti Mulay³⁵, Mohanasankar Sivaprakasam³⁵, Johannes C Paetzold⁴, Suprosanna Shit⁴, Ivan Ezhov⁴, Benedikt Wiestler³, Ben Glocker³⁶, Alexander Valentinitsch³, Markus Rempfler³⁷, Björn H Menze³⁸, Jan S Kirschke³

Affiliations

¹ Department of Informatics, Technical University of Munich, Germany; Munich School of BioEngineering, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany. Electronic address: anjany.sekuboyina@tum.de.
² Department of Informatics, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany.
³ Department of Neuroradiology, Klinikum Rechts der Isar, Germany.
⁴ Department of Informatics, Technical University of Munich, Germany.
⁵ Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Germany.
⁶ Institute of Computer Graphics and Vision, Graz University of Technology, Austria.
⁷ Gottfried Schatz Research Center: Biophysics, Medical University of Graz, Austria.
⁸ School of Computer Science, The University of Auckland, New Zealand.
⁹ Computer Vision Group, iFLYTEK Research South China, China.
¹⁰ Department of Radiology and Nuclear Medicine, Radboud University Medical Center Nijmegen, The Netherlands.
¹¹ Shenzhen Research Institute of Big Data, China.
¹² School of Biomedical Engineering, Health Science Center, Shenzhen University, China.
¹³ NVIDIA Corporation, USA.
¹⁴ Zuse Institute Berlin, Germany.
¹⁵ 1000shapes GmbH, Berlin, Germany.
¹⁶ Damo Academy, Alibaba Group, China.
¹⁷ Department of Mathematics, University of Innsbruck, Austria.
¹⁸ Department of Electronic Engineering, Fudan University, China; Department of Radiology, University of North Carolina at Chapel Hill, USA.
¹⁹ New York University, USA.
²⁰ EPITA Research and Development Laboratory (LRDE), France.
²¹ Deep Reasoning AI Inc, USA.
²² Chinese Academy of Sciences, China.
²³ Technical University of Munich, Germany.
²⁴ East China Normal University, China.
²⁵ Institute of Computing Technology, Chinese Academy of Sciences, China.
²⁶ Lenovo Group, China.
²⁷ Ping An Technologies, China.
²⁸ College of Computer Science and Technology, Zhejiang University, China; Real Doctor AI Research Centre, Zhejiang University, China.
²⁹ College of Computer Science and Technology, Zhejiang University, China.
³⁰ The University of Texas MD Anderson Cancer Center, USA.
³¹ Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, China.
³² Department of Computing, The Hong Kong Polytechnic University, China.
³³ The School of Biomedical Engineering, University of Technology Sydney, Australia.
³⁴ Department of Biomedical Engineering, Brno University of Technology, Czech Republic.
³⁵ Indian Institute of Technology Madras, India; Healthcare Technology Innovation Centre, India.
³⁶ Department of Computing, Imperial College London, UK.
³⁷ Friedrich Miescher Institute for Biomedical Engineering, Switzerland.
³⁸ Department of Informatics, Technical University of Munich, Germany; Department for Quantitative Biomedicine, University of Zurich, Switzerland.

PMID: 34340104
DOI: 10.1016/j.media.2021.102166

VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images

Anjany Sekuboyina et al. Med Image Anal. 2021 Oct.

. 2021 Oct:73:102166.

doi: 10.1016/j.media.2021.102166. Epub 2021 Jul 22.

Authors

Affiliations

¹ Department of Informatics, Technical University of Munich, Germany; Munich School of BioEngineering, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany. Electronic address: anjany.sekuboyina@tum.de.
² Department of Informatics, Technical University of Munich, Germany; Department of Neuroradiology, Klinikum Rechts der Isar, Germany.
³ Department of Neuroradiology, Klinikum Rechts der Isar, Germany.
⁴ Department of Informatics, Technical University of Munich, Germany.
⁵ Institute of Biological and Medical Imaging, Helmholtz Zentrum München, Germany.
⁶ Institute of Computer Graphics and Vision, Graz University of Technology, Austria.
⁷ Gottfried Schatz Research Center: Biophysics, Medical University of Graz, Austria.
⁸ School of Computer Science, The University of Auckland, New Zealand.
⁹ Computer Vision Group, iFLYTEK Research South China, China.
¹⁰ Department of Radiology and Nuclear Medicine, Radboud University Medical Center Nijmegen, The Netherlands.
¹¹ Shenzhen Research Institute of Big Data, China.
¹² School of Biomedical Engineering, Health Science Center, Shenzhen University, China.
¹³ NVIDIA Corporation, USA.
¹⁴ Zuse Institute Berlin, Germany.
¹⁵ 1000shapes GmbH, Berlin, Germany.
¹⁶ Damo Academy, Alibaba Group, China.
¹⁷ Department of Mathematics, University of Innsbruck, Austria.
¹⁸ Department of Electronic Engineering, Fudan University, China; Department of Radiology, University of North Carolina at Chapel Hill, USA.
¹⁹ New York University, USA.
²⁰ EPITA Research and Development Laboratory (LRDE), France.
²¹ Deep Reasoning AI Inc, USA.
²² Chinese Academy of Sciences, China.
²³ Technical University of Munich, Germany.
²⁴ East China Normal University, China.
²⁵ Institute of Computing Technology, Chinese Academy of Sciences, China.
²⁶ Lenovo Group, China.
²⁷ Ping An Technologies, China.
²⁸ College of Computer Science and Technology, Zhejiang University, China; Real Doctor AI Research Centre, Zhejiang University, China.
²⁹ College of Computer Science and Technology, Zhejiang University, China.
³⁰ The University of Texas MD Anderson Cancer Center, USA.
³¹ Department of Electronic and Information Engineering, The Hong Kong Polytechnic University, China.
³² Department of Computing, The Hong Kong Polytechnic University, China.
³³ The School of Biomedical Engineering, University of Technology Sydney, Australia.
³⁴ Department of Biomedical Engineering, Brno University of Technology, Czech Republic.
³⁵ Indian Institute of Technology Madras, India; Healthcare Technology Innovation Centre, India.
³⁶ Department of Computing, Imperial College London, UK.
³⁷ Friedrich Miescher Institute for Biomedical Engineering, Switzerland.
³⁸ Department of Informatics, Technical University of Munich, Germany; Department for Quantitative Biomedicine, University of Zurich, Switzerland.

PMID: 34340104
DOI: 10.1016/j.media.2021.102166

Abstract

Vertebral labelling and segmentation are two fundamental tasks in an automated spine processing pipeline. Reliable and accurate processing of spine images is expected to benefit clinical decision support systems for diagnosis, surgery planning, and population-based analysis of spine and bone health. However, designing automated algorithms for spine processing is challenging predominantly due to considerable variations in anatomy and acquisition protocols and due to a severe shortage of publicly available data. Addressing these limitations, the Large Scale Vertebrae Segmentation Challenge (VerSe) was organised in conjunction with the International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) in 2019 and 2020, with a call for algorithms tackling the labelling and segmentation of vertebrae. Two datasets containing a total of 374 multi-detector CT scans from 355 patients were prepared and 4505 vertebrae have individually been annotated at voxel level by a human-machine hybrid algorithm (https://osf.io/nqjyw/, https://osf.io/t98fz/). A total of 25 algorithms were benchmarked on these datasets. In this work, we present the results of this evaluation and further investigate the performance variation at the vertebra level, scan level, and different fields of view. We also evaluate the generalisability of the approaches to an implicit domain shift in data by evaluating the top-performing algorithms of one challenge iteration on data from the other iteration. The principal takeaway from VerSe: the performance of an algorithm in labelling and segmenting a spine scan hinges on its ability to correctly identify vertebrae in cases of rare anatomical variations. The VerSe content and code can be accessed at: https://github.com/anjany/verse.

Keywords: Labelling; Segmentation; Spine; Vertebrae.

PubMed Disclaimer

Conflict of interest statement

Declaration of Competing Interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Publication types

Actions

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions

Grants and funding

L30 CA274754/CA/NCI NIH HHS/United States

LinkOut - more resources

Full Text Sources
- Elsevier Science
- Ovid Technologies, Inc.
Other Literature Sources
- The Lens - Patent Citations Database
Medical
- MedlinePlus Consumer Health Information
- MedlinePlus Health Information
Miscellaneous
- NCI CPTAC Assay Portal

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images

Affiliations

VerSe: A Vertebrae labelling and segmentation benchmark for multi-detector CT images

Authors

Affiliations

Abstract

Conflict of interest statement

Publication types

MeSH terms

Grants and funding

LinkOut - more resources

Full Text Sources

Other Literature Sources

Medical

Miscellaneous