MiNuGAN: Dual Segmentation of Mitoses and Nuclei Using Conditional GANs on Multi-center Breast H&E Images

Abstract

Breast cancer is the second most commonly diagnosed type of cancer among women as of 2021. Grading of histopathological images is used to guide breast cancer treatment decisions and a critical component of this is a mitotic score, which is related to tumor aggressiveness. Manual mitosis counting is an extremely tedious manual task, but automated approaches can be used to overcome inefficiency and subjectivity. In this paper, we propose an automatic mitosis and nuclear segmentation method for a diverse set of H&E breast cancer pathology images. The method is based on a conditional generative adversarial network to segment both mitoses and nuclei at the same time. Architecture optimizations are investigated, including hyper parameters and the addition of a focal loss. The accuracy of the proposed method is investigated using images from multiple centers and scanners, including TUPAC16, ICPR14 and ICPR12 datasets. In TUPAC16, we use 618 carefully annotated images of size 256×256 scanned at 40×. TUPAC16 is used to train the model, and segmentation performance is measured on the test set for both nuclei and mitoses. Results on 200 held-out testing images from the TUPAC16 dataset were mean DSC = 0.784 and 0.721 for nuclear and mitosis, respectively. On 202 ICPR12 images, mitosis segmentation accuracy had a mean DSC = 0.782, indicating the model generalizes well to unseen datasets. For datasets that had mitosis centroid annotations, which included 200 TUPAC16, 202 ICPR12 and 524 ICPR14, a mean F1-score of 0.854 was found indicating high mitosis detection accuracy.

Keywords: Breast cancer; Computer-aided detection; Focal loss; Generative adversarial network; Mitosis detection; Semantic segmentation.

Figure 1

Mitosis (green) and nuclear (red) annotations from TUPAC16.

Figure 2

MiNuGAN for dual mitosis and nuclear segmentation in H&E breast cancer images.

Figure 3

DSC distributions for cGAN configurations in Table 2.

Figure 4

Mean DSC and DSC coefficient of variation (CoV) for cGAN configurations.

Figure 5

Mitosis segmentation performance for TUPAC16 and ICPR12.

Figure 6

cGAN segmentation results with and without focal loss. a) original images, b) ground truth, c) configuration 1 (baseline without focal loss) and d) configuration 4 (baseline with focal loss).

Figure 7

DSC distributions for mitoses and nuclei for MiNuGAN, UNET, UNET/focal.

Figure 8

Mean DSC and DSC coefficient of variation for MiNuGAN, UNET, UNET/focal.

Figure 9

Segmentation results using three different models. a) input image, b) ground truth, c) MiNUGAN d) U-Net e) U-Net with focal loss.

Figure 10

Segmentation and detection of mitosis using MiNuGAN for a) ICPR12, b) ICPR14, c) TUPAC16. Segmented mitoses are shown in green and nuclei in red. False positives are denoted by a yellow box, and false negatives by an orange box.