Generative Adversarial Network Based Automatic Segmentation of Corneal Subbasal Nerves on In Vivo Confocal Microscopy Images

Abstract

Purpose: In vivo confocal microscopy (IVCM) is a noninvasive, reproducible, and inexpensive diagnostic tool for corneal diseases. However, widespread and effortless image acquisition in IVCM creates serious image analysis workloads on ophthalmologists, and neural networks could solve this problem quickly. We have produced a novel deep learning algorithm based on generative adversarial networks (GANs), and we compare its accuracy for automatic segmentation of subbasal nerves in IVCM images with a fully convolutional neural network (U-Net) based method.

Methods: We have collected IVCM images from 85 subjects. U-Net and GAN-based image segmentation methods were trained and tested under the supervision of three clinicians for the segmentation of corneal subbasal nerves. Nerve segmentation results for GAN and U-Net-based methods were compared with the clinicians by using Pearson's R correlation, Bland-Altman analysis, and receiver operating characteristics (ROC) statistics. Additionally, different noises were applied on IVCM images to evaluate the performances of the algorithms with noises of biomedical imaging.

Results: The GAN-based algorithm demonstrated similar correlation and Bland-Altman analysis results with U-Net. The GAN-based method showed significantly higher accuracy compared to U-Net in ROC curves. Additionally, the performance of the U-Net deteriorated significantly with different noises, especially in speckle noise, compared to GAN.

Conclusions: This study is the first application of GAN-based algorithms on IVCM images. The GAN-based algorithms demonstrated higher accuracy than U-Net for automatic corneal nerve segmentation in IVCM images, in patient-acquired images and noise applied images. This GAN-based segmentation method can be used as a facilitating diagnostic tool in ophthalmology clinics.

Translational relevance: Generative adversarial networks are emerging deep learning models for medical image processing, which could be important clinical tools for rapid segmentation and analysis of corneal subbasal nerves in IVCM images.

Figure 1.

Example IVCM images and masks for U-Net and GAN-based segmentation methods.

Figure 2.

U-Net structure.

Figure 3.

General Conditional Generative Adversarial Network structure used in the study. Blue mask generated by the generator is concatenated with the input image and labeled as fake, while the green mask is obtained by experts and labelled as real.

Figure 4.

Generator part of GAN structure.

Figure 5.

PatchGAN structure used for the discriminator part for GAN.

Figure 6.

Correlation (a) and Bland Altman (b) plots for U-Net structure.

Figure 7.

Correlation (a) and Bland Altman (b) plots for GAN structure.

Figure 8.

ROC curves for GAN and U-Net structures.

Figure 9.

ROC curves for U-Net (a) and GAN (b) structures with different noises.