Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Sep 8;16(9):e1008179.
doi: 10.1371/journal.pcbi.1008179. eCollection 2020 Sep.

Detection and segmentation of morphologically complex eukaryotic cells in fluorescence microscopy images via feature pyramid fusion

Nikolaus Korfhage  1 , Markus Mühling  1 , Stephan Ringshandl  2 , Anke Becker  2 , Bernd Schmeck  3   4 , Bernd Freisleben  1   2
Affiliations

Detection and segmentation of morphologically complex eukaryotic cells in fluorescence microscopy images via feature pyramid fusion

Nikolaus Korfhage et al. PLoS Comput Biol. .

Abstract

Detection and segmentation of macrophage cells in fluorescence microscopy images is a challenging problem, mainly due to crowded cells, variation in shapes, and morphological complexity. We present a new deep learning approach for cell detection and segmentation that incorporates previously learned nucleus features. A novel fusion of feature pyramids for nucleus detection and segmentation with feature pyramids for cell detection and segmentation is used to improve performance on a microscopic image dataset created by us and provided for public use, containing both nucleus and cell signals. Our experimental results indicate that cell detection and segmentation performance significantly benefit from the fusion of previously learned nucleus features. The proposed feature pyramid fusion architecture clearly outperforms a state-of-the-art Mask R-CNN approach for cell detection and segmentation with relative mean average precision improvements of up to 23.88% and 23.17%, respectively.

PubMed Disclaimer

Conflict of interest statement

The authors have declared that no competing interests exist.

Figures

Fig 1
Fig 1. Image with ground truth segmentation.
Nucleus signal (a), cytoplasm signal (b), and ground truth segmentation (c).
Fig 2
Fig 2. Feature pyramid fusion of nucleus features.
Pre-trained nucleus features (violet) are fused with features of the feature pyramid in the cell detection and segmentation model (green) by either concatenation or addition.
Fig 3
Fig 3. Including nucleus information for cell segmentation.
(a) without nucleus information, (b) with additional input for the nucleus channel, and (c) with fused nucleus features.
Fig 4
Fig 4. Weights for instances.
For each box proposal, crops are resized to 28 × 28 pixels. Crop from the input image (a), full-image segmentation mask (b), cell mask (c) and weight matrix (d).
Fig 5
Fig 5. Visualization of cell segmentation errors on a 256 × 256 patch of clustered cells in test data: Predicted masks (red) differ only slightly from ground truth masks (white).
Fig 6
Fig 6. APs for cell segmentation on clustered cells.
Fig 7
Fig 7. Visualization of segmentation of clustered cells.
Top: nucleus signal (a), cytoplasm signal (b), and ground truth segmentation (c). Bottom: Instance segmentations predicted by for model without nucleus information (d), with nucleus channel (e), and FPF ⊕ with weighted loss (f).
Fig 8
Fig 8. Cell segmentation of clustered cells by Feature Pyramid Fusion (FPF) on a 512 × 512 patch.
See this image and copyright information in PMC

References

    1. Reuter JA, Spacek DV, Snyder MP. High-throughput sequencing technologies. Molecular Cell. 2015;58(4):586–597. 10.1016/j.molcel.2015.05.004 - DOI - PMC - PubMed
    1. Usaj MM, Styles EB, Verster AJ, Friesen H, Boone C, Andrews BJ. High-content screening for quantitative cell biology. Trends in Cell Biology. 2016;26(8):598–611. - PubMed
    1. Chen X, Zhu Y, Li F, Zheng ZY, Chang EC, Ma J, et al. Accurate segmentation of touching cells in multi-channel microscopy images with geodesic distance based clustering. Neurocomputing. 2015;149:39–47.
    1. He K, Gkioxari G, Dollár P, Girshick R. Mask R-CNN. In: IEEE Int. Conf on Computer Vision (ICCV). IEEE; 2017. p. 2980–2988.
    1. Al-Kofahi Y, Lassoued W, Lee W, Roysam B. Improved automatic detection and segmentation of cell nuclei in histopathology images. IEEE Transactions on Biomedical Engineering. 2010;57(4):841–852. - PubMed

Publication types