Deep Adversarial Training for Multi-Organ Nuclei Segmentation in Histopathology Images

Abstract

Nuclei mymargin segmentation is a fundamental task for various computational pathology applications including nuclei morphology analysis, cell type classification, and cancer grading. Deep learning has emerged as a powerful approach to segmenting nuclei but the accuracy of convolutional neural networks (CNNs) depends on the volume and the quality of labeled histopathology data for training. In particular, conventional CNN-based approaches lack structured prediction capabilities, which are required to distinguish overlapping and clumped nuclei. Here, we present an approach to nuclei segmentation that overcomes these challenges by utilizing a conditional generative adversarial network (cGAN) trained with synthetic and real data. We generate a large dataset of H&E training images with perfect nuclei segmentation labels using an unpaired GAN framework. This synthetic data along with real histopathology data from six different organs are used to train a conditional GAN with spectral normalization and gradient penalty for nuclei segmentation. This adversarial regression framework enforces higher-order spacial-consistency when compared to conventional CNN models. We demonstrate that this nuclei segmentation approach generalizes across different organs, sites, patients and disease states, and outperforms conventional approaches, especially in isolating individual and overlapping nuclei.

Fig. 1.

(a) Figure showing full-slide images, 1000×1000 cropped images and sparse stain normalization for nine different organs sources from the NIH TCGA database. (b) Unpaired synthetic data generation using randomly-generated polygon masks mapped to H&E images. The architecture includes a dual-GAN setup with cycle-consistency loss. Two generators learn mappings, G, and S between a mask (M) and a histology image (N) G : M → N and S : N → M and two discriminators classify the pairs of M and N as real or fake. (c) The conditional GAN setup for segmenting nuclei. The discriminator enhances the receptive field of the generator while learning an efficient loss function for the task.

Fig. 2.

Representative patches from six different organs and corresponding nuclei segmentation masks predicted by our proposed method, overlaid on ground truth segmentation masks. The green region represents an overlap between the prediction and manually labeled ground truth whereas the red region represents a disparity between the two. The predominance of the green region demonstrates accurate labeling. The bar charts compare the AJI for all test patches of corresponding organs with state-of-the-art methods (DIST [34] and CNN-3C [31]) as well as commonly-used segmentation tools in Fiji [80] and Cell Profiler [81].