Deep learning for automated boundary detection and segmentation in organ donation photography

Abstract

Objectives: Medical photography is ubiquitous and plays an increasingly important role in the fields of medicine and surgery. Any assessment of these photographs by computer vision algorithms requires first that the area of interest can accurately be delineated from the background. We aimed to develop deep learning segmentation models for kidney and liver organ donation photographs where accurate automated segmentation has not yet been described.

Methods: Two novel deep learning models (Detectron2 and YoloV8) were developed using transfer learning and compared against existing tools for background removal (macBGRemoval, remBGisnet, remBGu2net). Anonymised photograph datasets comprised training/internal validation sets (821 kidney and 400 liver images) and external validation sets (203 kidney and 208 liver images). Each image had two segmentation labels: whole organ and clear view (parenchyma only). Intersection over Union (IoU) was the primary outcome, as the recommended metric for assessing segmentation performance.

Results: In whole kidney segmentation, Detectron2 and YoloV8 outperformed other models with internal validation IoU of 0.93 and 0.94, and external validation IoU of 0.92 and 0.94, respectively. Other methods - macBGRemoval, remBGisnet and remBGu2net - scored lower, with highest internal validation IoU at 0.54 and external validation at 0.59. Similar results were observed in liver segmentation, where Detectron2 and YoloV8 both showed internal validation IoU of 0.97 and external validation of 0.92 and 0.91, respectively. The other models showed a maximum internal validation and external validation IoU of 0.89 and 0.59 respectively. All image segmentation tasks with Detectron2 and YoloV8 completed within 0.13-1.5 s per image.

Conclusions: Accurate, rapid and automated image segmentation in the context of surgical photography is possible with open-source deep-learning software. These outperform existing methods and could impact the field of surgery, enabling similar advancements seen in other areas of medical computer vision.

Keywords: artificial intelligence; computer vision; deep learning; image segmentation; transplant surgery.

Figure 1

Median intersection over union (IoU) performance across segmentation methods for kidney and liver retrieval photographs in test and external validation datasets. Includes representative original images, without ground truth annotations, and segmentation outcomes for each method.

Figure 2

Comparative analysis of segmentation model performance with intersection over union (IoU), dice coefficient (DSC) and area under the receiver operating characteristic curve (AUROC). Box plots summarise the performance of the segmentation models across kidney and liver images.

Figure 3

Receiver operating characteristic (ROC) curves comparing the performance of the image segmentation methods for object detection in retrieval photographs. Performance metrics are indicated for both internal and external validation phases, with ‘chance’ representing a baseline random classifier.