Label-free cell segmentation of diverse lymphoid tissues in 2D and 3D

Abstract

Unlocking and quantifying fundamental biological processes through tissue microscopy requires accurate, in situ segmentation of all cells imaged. Currently, achieving this is complex and requires exogenous fluorescent labels that occupy significant spectral bandwidth, increasing the duration and complexity of imaging experiments while limiting the number of channels remaining to address the study's objectives. We demonstrate that the excitation light reflected during routine confocal microscopy contains sufficient information to achieve accurate, label-free cell segmentation in 2D and 3D. This is achieved using a simple convolutional neural network trained to predict the probability that reflected light pixels belong to either nucleus, cytoskeleton, or background classifications. We demonstrate the approach across diverse lymphoid tissues and provide video tutorials demonstrating deployment in Python and MATLAB or via standalone software for Windows.

Keywords: 2D; 3D; cell segmentation; confocal microscopy; digital pathology; immunofluorescence; label free; quantitative; single-cell; tissue.

Graphical abstract

Figure 1

Label-free cell segmentation of tissue microscopy image data collected by routine confocal microscopy (A–D) Image data (here, mouse splenic tissue) for initial network training are obtained from serial tissue sections stained for (A) nuclei (Hoechst 33342) and (B) cytoskeletal f-actin (phalloidin-AlexaFluor 647) while simultaneously collecting (C and D) reflected laser excitation light by detector placement close (± 5 nm) to the excitation wavelength. (E) Binary pixel-classification labels representing “background,” “nuclei,” and “cytoskeleton” classes are created by thresholding the fluorescence data. (F) A neural network using a simple U-Net architecture is trained to output the probability that pixels in the reflectance image belong to each of these classes. (A)–(F) show zoomed insets of the exact same image region. Comparing across these insets, the outputted probability maps (F) exhibit consistent intensities across each image field, with clear gradients that flow between the individual classifications. This enables easy, consistent instance segmentations of individual cell objects using routine watershed approaches. (G and H) For subsequent slides, nuclei and actin stains are no longer required as the cell segmentation is achieved direct from the reflectance information via the probability map images. This establishes the cell segmentation while leaving the entire detection spectrum free for fluorescence-based analyses. For example, (H) shows the approach operating with CD3-eFluor450, CD4-PE, and CD11c-eFluor660 immunofluorescence conjugates utilizing the spectral bandwidth previously occupied by the nuclei (Hoechst 33342) and actin (phalloidin-AlexaFluor 647) stains. The label-free cell segmentation is overlaid. (H) Insets demonstrate successful label-free cell segmentation of both CD marker-stained and entirely unstained cells in both red (green inset) and white (gray inset) pulp tissue regions. (A–H) Main image scale bars: 250 μm, and inset image scale bars: 10 μm.

Figure 2

Assessing cell segmentation accuracies using mouse Peyer’s patch tissue (A) Hand-drawn cell segmentation performed using nuclei/actin fluorescence information for the tissue region indicated by the yellow box in the wider, tile-scanned image. (B and C) Automated cell instance segmentations for the same image-region as (A) using either (B) the raw nuclei and actin fluorescence data or (C) the label-free probability maps obtained from the neural network using reflectance alone as input (image data from this tissue section were unseen during training). (D and E) Cell-object intersection-over-union (IOU) score distributions comparing the hand-drawn cell segmentations shown in (A) against the automated cell segmentations shown in (B and C) using either (D) fluorescence or (E) label-free information. (F) Example hand-drawn versus label-free cell segmentation comparisons and IOU scores. The positions of each cell in the source images are shown by the cell-object numberings in (A), (C), and (F). An IOU score of 1 represents perfect per-pixel overlap between hand-drawn and automated cell segmentations. (F) Within the comparison presented here, scores ≥0.6 are seen to represent a good match, approaching the limit of hand-drawing accuracy given the relatively low resolution of the source image data. (A–C) Scale bars, 100 μm. (F) Scale bar, 10 μm.

Figure 3

Label-free cell segmentation enables image-based cell profiling (A) Tile-scanned mouse Peyer’s patch tissue section imaged for reflectance in addition to immunofluorescence markers for CD11c (i.e., mononuclear phagocyte antigen-presenting cells) and CD3 (T lymphocytes). The yellow region of interest (ROI) represents the lymphoid tissue upon which the label-free cell segmentation approach was deployed (∼16,000 cells). Outside of the ROI, the reflectance image is seen to still provide interpretable histological context. (B–E) Flow cytometry-type gating to establish CD3⁺ and CD11c⁺ cell populations informed by secondary only, fluorescence-minus-one (fmo) and isotype single-cell fluorescence distributions obtained from label-free cell object data collected from adjacent, serial tissue sections. Due to the dense cellular packing of lymphoid tissue, (C and E) second sequential gates on the fluorescence area occupied per cell object helped to reduce (F) bystander-positive events caused by fluorescence overlap into neighboring cells. (G) Cell map view showing the gated cell populations in situ using flood filling of label-free cell-objects. Juxtaposed CD11c-CD3 neighboring cells that still identified positive for both CD markers after bystander removal are shown in white. (H and I) CD11c and CD3 expression maps with cell objects shaded into four levels (dim, low, intermediate, high) according to each segmented cell’s level of immunofluorescence. (J–L) Nearest-cell-neighbor maps simplifying the view shown in (A) to only show touching groups of cell objects according the combinations (J) CD11c⁺-CD3⁺ (i.e., APC-T), (K) CD11c⁺-CD3^-/CD11c⁻ (i.e., APC-B), and (L) CD3⁺-CD3⁻/CD11c⁻ (i.e., T-B). In this way, the views give a sense of key cell types within interactive distances of one another. The dashed line in (J) and (K) indicates the subepithelial dome tissue region. Scale bars: 500 μm.

Figure 4

3D label-free cell segmentation of tissue microscopy image data collected by routine confocal microscopy (A–H) Stepwise exemplification of the 3D strategy using z stack image data of mouse mesenteric lymph node tissue. Outcomes at each step are displayed by (A, C, and G) orthoslice and (B, D, and H) 3D volumetric projection views, with the latter cut away to better display outcomes along the z dimension. (A and B) 3D reflectance signal. (C–F) Label-free probability maps outputted from a 3D U-Net neural network (C and D) with insets demonstrating the (E) label-free probability map representation for the cytoskeleton compared with (F) fluorescent cytoskeletal F-actin staining. (G and H) 3D label-free cell segmentation results where (repeated) filled colors represent individual cell objects. (I–N) Validation of the 3D label-free approach using tissue sections immunolabelled for FOXP3 or matched isotype control with no nuclei or actin staining present. (J and K) Flow cytometry-type gating using cell-object fluorescence distributions to establish FOXP3⁺ events from cell intensity and fluorescence volume information. (L) 3D projection of the label-free cell segmentation results. On the left, gated FOXP3⁺ cells are identified using red surface overlays. On the right, both FOXP3⁺ events (red) and their touching nearest-cell neighbors (cyan) are shown in situ. (M), 3D projections of individual FOXP3⁺ cell objects cut out and montaged from the label-free segmentation. An intranuclear core of FOXP3 staining is visible surrounded by the label-free probability map for the cytoskeleton classification. (N) 3D projections of individual FOXP3⁺ cell objects and their touching nearest-cell neighbors cut out and montaged using the label-free cell segmentation. (A, C, and G) Scale bars: 50 μm.