Direct three-dimensional segmentation of prostate glands with nnU-Net

Abstract

Significance: In recent years, we and others have developed non-destructive methods to obtain three-dimensional (3D) pathology datasets of clinical biopsies and surgical specimens. For prostate cancer risk stratification (prognostication), standard-of-care Gleason grading is based on examining the morphology of prostate glands in thin 2D sections. This motivates us to perform 3D segmentation of prostate glands in our 3D pathology datasets for the purposes of computational analysis of 3D glandular features that could offer improved prognostic performance.

Aim: To facilitate prostate cancer risk assessment, we developed a computationally efficient and accurate deep learning model for 3D gland segmentation based on open-top light-sheet microscopy datasets of human prostate biopsies stained with a fluorescent analog of hematoxylin and eosin (H&E).

Approach: For 3D gland segmentation based on our H&E-analog 3D pathology datasets, we previously developed a hybrid deep learning and computer vision-based pipeline, called image translation-assisted segmentation in 3D (ITAS3D), which required a complex two-stage procedure and tedious manual optimization of parameters. To simplify this procedure, we use the 3D gland-segmentation masks previously generated by ITAS3D as training datasets for a direct end-to-end deep learning-based segmentation model, nnU-Net. The inputs to this model are 3D pathology datasets of prostate biopsies rapidly stained with an inexpensive fluorescent analog of H&E and the outputs are 3D semantic segmentation masks of the gland epithelium, gland lumen, and surrounding stromal compartments within the tissue.

Results: nnU-Net demonstrates remarkable accuracy in 3D gland segmentations even with limited training data. Moreover, compared with the previous ITAS3D pipeline, nnU-Net operation is simpler and faster, and it can maintain good accuracy even with lower-resolution inputs.

Conclusions: Our trained DL-based 3D segmentation model will facilitate future studies to demonstrate the value of computational 3D pathology for guiding critical treatment decisions for patients with prostate cancer.

Keywords: biomedical image processing; computational three-dimensional pathology; deep learning; gland segmentation; prostate cancer.

Fig. 1

General pipeline comparison between ITAS3D (the upper route) and nnU-Net (the lower route) for 3D prostate gland segmentation.

Fig. 2

Inputs for training an nnU-Net model from 3D H&E images and paired segmentation masks generated by ITAS3D.

Fig. 3

Qualitative evaluation of the trained model’s performance. (a), (b) 2D frames showing side-by-side comparisons between nnU-Net-generated segmentation masks and ITAS3D-generated segmentation masks, (Videos 1 and 2 show 3D datasets of the masks) both from the same H&E image input. The examples shown in panels (a) and (b) are from different tissue samples. Bold white arrows point to regions where nnU-Net outperforms ITAS3D. $\text{Scale bars}=100\mu \mathrm{m}$ (Video 1, MP4, 10.9 MB [URL: https://doi.org/10.1117/1.JBO.29.3.036001.s1]; Video 2, MP4, 10.9 MB [URL: https://doi.org/10.1117/1.JBO.29.3.036001.s2]).

Fig. 4

Quantitative measurements of the nnU-Net model’s performance in terms of Dice coefficient and 3D Hausdorff distance as calculated from 10 manually annotated test regions (3D volumetric depth stacks each containing hundreds of manually annotated 2D images) that were not used for training. Asterisk (*) denotes a $p\text{\hspace{0.17em}value}<0.05$. (a) Example of a nnU-Net generated segmentation mask versus the manually annotated segmentation mask (Video 3, MP4, 11 MB [URL: https://doi.org/10.1117/1.JBO.29.3.036001.s3]). (b) Benchmark of the model when trained for 100, 200, 300, 500, and 1000 epochs, respectively. (c) Benchmark of nnU-Net method against ITAS3D and other baseline segmentation methods. (d) Benchmark of the original nnU-Net model against a new nnU-Net model trained on datasets with 2X-higher resolution (8X larger size for a 3D dataset).

Fig. 5

Speed benchmark between nnU-Net and ITAS3D execution with the same PC workstation (see Sec. 2). The ITAS3D timeline excludes the time taken for manual parameter adjustments, which often makes ITAS3D much more time consuming than plotted here. The average physical size of the biopsies used for these benchmarking tests was approximately $1\times 0.7\times 20\mathrm{mm}$.