Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Dec;37(6):3208-3216.
doi: 10.1007/s10278-024-01138-2. Epub 2024 Jun 6.

MedYOLO: A Medical Image Object Detection Framework

Joseph Sobek  1 , Jose R Medina Inojosa  2   3 , Betsy J Medina Inojosa  2 , S M Rassoulinejad-Mousavi  4 , Gian Marco Conte  4 , Francisco Lopez-Jimenez  2 , Bradley J Erickson  4
Affiliations

MedYOLO: A Medical Image Object Detection Framework

Joseph Sobek et al. J Imaging Inform Med. 2024 Dec.

Abstract

Artificial intelligence-enhanced identification of organs, lesions, and other structures in medical imaging is typically done using convolutional neural networks (CNNs) designed to make voxel-accurate segmentations of the region of interest. However, the labels required to train these CNNs are time-consuming to generate and require attention from subject matter experts to ensure quality. For tasks where voxel-level precision is not required, object detection models offer a viable alternative that can reduce annotation effort. Despite this potential application, there are few options for general-purpose object detection frameworks available for 3-D medical imaging. We report on MedYOLO, a 3-D object detection framework using the one-shot detection method of the YOLO family of models and designed for use with medical imaging. We tested this model on four different datasets: BRaTS, LIDC, an abdominal organ Computed tomography (CT) dataset, and an ECG-gated heart CT dataset. We found our models achieve high performance on a diverse range of structures even without hyperparameter tuning, reaching mean average precision (mAP) at intersection over union (IoU) 0.5 of 0.861 on BRaTS, 0.715 on the abdominal CT dataset, and 0.995 on the heart CT dataset. However, the models struggle with some structures, failing to converge on LIDC resulting in a mAP@0.5 of 0.0.

Keywords: Computed tomography; Convolutional neural network; Deep learning; Magnetic resonance; Medical imaging; Object detection.

PubMed Disclaimer

Conflict of interest statement

Declarations. Ethics Approval: This retrospective study was reviewed and deemed exempt by Mayo Clinic’s Institutional Review Board. Consent to Participate: The informed consent was waived by ethics groups. Consent for Publication: The informed consent was waived by ethics groups. Competing Interests: The authors declare no competing interests.

Figures

Fig. 1
Fig. 1
Architectural diagram of MedYOLO and its component modules. Due to its complexity, the Detect module is easier understood through code and not displayed here
Fig. 2
Fig. 2
Bounding box segmentations for BRaTS validation data using MedYOLO-S on three different scans
Fig. 3
Fig. 3
Bounding box segmentations for the ECG-gated cardiovascular CT dataset. Segmentations correspond to the following: ground truth (a), MedYOLO prediction (b), nnDetection highest confidence prediction (c), nnDetection 2nd highest confidence prediction (d), nnDetection 3rd highest confidence prediction (e)
Fig. 4
Fig. 4
Bounding box segmentations for several organs in the abdominal dataset using MedYOLO-L. a Left and right kidneys. b Spleen. c Pancreas. d Diaphragm. e Bladder. f Aorta. g Stomach. h Liver
See this image and copyright information in PMC

References

    1. Baumgartner M., Jäger P.F., Isensee F., Maier-Hein K.H.: nnDetection: A Self-configuring Method for Medical Object Detection. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. 10.1007/978-3-030-87240-3_51, 2021
    1. Redmon J, Farhadi A: YOLO9000: Better, Faster, Stronger. 2017 IEEE Conference on Computer Vision and Pattern Recognition. 10.48550/arXiv.1612.08242, 2016
    1. Jocher G, et al.: ultralytics/yolov5: v7.0 - YOLOv5 SOTA Realtime Instance Segmentation. 10.5281/zenodo.3908559, 2022
    1. U. Baid, et al.: The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification. 10.48550/arXiv.2107.02314, 2021
    1. B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al.: The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS). IEEE Transactions on Medical Imaging 34:1993-2024, 2015 - PMC - PubMed

MeSH terms