Fast segmentation of stained nuclei in terabyte-scale, time resolved 3D microscopy image stacks

Abstract

Automated analysis of multi-dimensional microscopy images has become an integral part of modern research in life science. Most available algorithms that provide sufficient segmentation quality, however, are infeasible for a large amount of data due to their high complexity. In this contribution we present a fast parallelized segmentation method that is especially suited for the extraction of stained nuclei from microscopy images, e.g., of developing zebrafish embryos. The idea is to transform the input image based on gradient and normal directions in the proximity of detected seed points such that it can be handled by straightforward global thresholding like Otsu's method. We evaluate the quality of the obtained segmentation results on a set of real and simulated benchmark images in 2D and 3D and show the algorithm's superior performance compared to other state-of-the-art algorithms. We achieve an up to ten-fold decrease in processing times, allowing us to process large data sets while still providing reasonable segmentation results.

Figure 1. Processing steps for the generation of a LoG scale-space maximum intensity projection used for 2D seed detection.

Original image (A), LoG filtered image with and (B, C), LoG scale-space maximum intensity projection with , and (D) and the detected seeds plotted on the original image (E).

Figure 2. Processing steps that are performed in parallel for each detected seed point.

Cropped raw image (A), Gaussian smoothed left-right derivative image (B), dot product of the normalized gradient with the seed normal (C), raw image with smoothed gradient and normal vector field overlay (D), weighted version of the previously calculated dot product (E), resulting intensity image (F) and the final segmentation result (G).

Figure 3. Exemplary weighting kernel for depicted in 1D (A) and 2D (B).

The kernel should be chosen such that the region of interest yields high weights.

Figure 4. Comparison of the segmentation quality achieved by the investigated algorithms on 2D benchmark images from the Broad Bioimage Benchmark Collection (BBBC006v1).

Original image (A), adaptive thresholding using Otsu’s method (B), Otsu’s method combined with watershed-based blob splitting , (C), geodesic active contours (D), gradient vector flow tracking (E), graph-cuts segmentation (F), TWANG segmentation (G) and a false colored original image (H). The symbols indicate segmentation errors for nuclei that are either split (#), merged (+), missing (o) or spurious (∼).

Figure 5. Comparison of the segmentation quality achieved by the investigated algorithms on simulated 3D benchmark images by Svoboda et al. (HL60 cell line, low SNR, 75% clustering probability).

Simulated original image (A), adaptive thresholding using Otsu’s method (B), Otsu’s method combined with watershed-based blob splitting , (C), geodesic active contours (D), gradient vector flow tracking (E), graph-cuts segmentation (F), TWANG segmentation (G) and the simulated ground truth image (H). The symbols indicate segmentation errors for nuclei that are either split (#), merged (+), missing (o) or spurious (∼).

Figure 6. Bar plot of the measured processing times in seconds (lower values are better).

Image sizes correspond to 256×256×50 (S), 512×512×100 (M), 1024×1024×200 (L) and 2048×2048×400 (XL) voxels. Missing bars indicate that the respective algorithms failed to process the given image size. TWANG segmentation turned out to be the fastest algorithm in all tested categories and was the only method that was able to process the XL images with the given memory constraint of 32 GB.

Figure 7. Comparison of the segmentation quality achieved by the investigated algorithms on a 3D image of labeled nuclei of a zebrafish embryo acquired using DSLM.

The panels show the maximum intensity projection of 3 neighbouring z-slices. Original image (A), adaptive thresholding using Otsu’s method (B), Otsu’s method combined with watershed-based blob splitting , (C), geodesic active contours (D), gradient vector flow tracking (E), graph-cuts segmentation (F), TWANG segmentation (G) and a false colored original image (H). The symbols indicate segmentation errors for nuclei that are either split (#), merged (+), missing (o) or spurious (∼).

Figure 8. Results of the TWANG segmentation pipeline applied on two images of a developing zebrafish embryo.

The images were captured at the 78000 cells (A,B) and at the 11 hpf stage with 12000 cells (C,D), respectively. The panels show maximum intensity projections of the raw images (A,C) and the resulting segmentation using our TWANG segmentation pipeline (B,D). Each 3D image stack has a file size of GB and comprises 2560×2160×500 voxels with a dynamic range of 16 bits. Processing one image stack takes approximately minutes on a common desktop machine, depending on the developmental stage of the embryo. Typical experiments may be comprised of up to z-stacks (TB) for the spatio-temporal analysis of a single embryo.