A fully-automated, robust, and versatile algorithm for long-term budding yeast segmentation and tracking

Abstract

Live cell time-lapse microscopy, a widely-used technique to study gene expression and protein dynamics in single cells, relies on segmentation and tracking of individual cells for data generation. The potential of the data that can be extracted from this technique is limited by the inability to accurately segment a large number of cells from such microscopy images and track them over long periods of time. Existing segmentation and tracking algorithms either require additional dyes or markers specific to segmentation or they are highly specific to one imaging condition and cell morphology and/or necessitate manual correction. Here we introduce a fully automated, fast and robust segmentation and tracking algorithm for budding yeast that overcomes these limitations. Full automatization is achieved through a novel automated seeding method, which first generates coarse seeds, then automatically fine-tunes cell boundaries using these seeds and automatically corrects segmentation mistakes. Our algorithm can accurately segment and track individual yeast cells without any specific dye or biomarker. Moreover, we show how existing channels devoted to a biological process of interest can be used to improve the segmentation. The algorithm is versatile in that it accurately segments not only cycling cells with smooth elliptical shapes, but also cells with arbitrary morphologies (e.g. sporulating and pheromone treated cells). In addition, the algorithm is independent of the specific imaging method (bright-field/phase) and objective used (40X/63X/100X). We validate our algorithm's performance on 9 cases each entailing a different imaging condition, objective magnification and/or cell morphology. Taken together, our algorithm presents a powerful segmentation and tracking tool that can be adapted to numerous budding yeast single-cell studies.

Fig 1. Automated seeding overview.

(A) Example phase image. (B) First step of automated seeding algorithm: Pre-processing and watershed. In this step, the watershed transform is applied to the processed image. (C) Phase image with watershed lines (yellow). (D) Flowchart of the second step of automated seeding: Automated correction and fine-tuning. At this step, the cell boundaries are automatically fine-tuned, and segmentation errors are automatically corrected. (E) The result of the automated seeding step. Each cell boundary is marked with a different color.

Fig 2. Automated correction & fine-tuning step examples.

(A) Refining cell boundaries: The watershed lines do not mark the exact cell boundaries (first column, magenta). Our algorithm automatically fine-tunes these watershed lines and marks the correct cell boundary (third column, red). (B) Under-segmentation correction: Sometimes the watershed lines merge multiple cells (first column, magenta). Such mistakes are detected and corrected automatically (fourth column, red). (C) Over-segmentation correction: Sometimes the watershed lines divide a cell into multiple pieces (first column). After applying the segmentation subroutine several times, each piece converges towards the correct cell segmentation and thus the pieces overlap significantly (fourth column). If the overlap between two pieces are above a certain threshold, then they are merged (fifth column, red). (D) Distribution of Overlaps: The algorithm sometimes assigns the same pixels to the segmentations of adjacent cells (Also see section Distribution of overlapping initial segmentations), which leads to overlapping cell segmentations. Such overlaps (fourth column, yellow) are distributed among the cells based on their scores.

Fig 3. Time gain, speedup and efficiency achieved by parallelization.

An example field of view imaged over 10 hours (200 time points, 360 cells at the last time point) was segmented sequentially and in parallel with varying number of workers. (A) Runtimes. (B) Speedup is calculated by dividing the sequential execution time by the parallel execution time. With 40 workers the algorithm runs 15.4 times faster. (C) Efficiency is the speedup per processor. Note that the efficiency goes down as the number of processors increases.

Fig 4. Distribution of overlapping initial segmentations.

(A) Example phase image showing two neighboring cells: There is a bright halo (phase halo) around the cells in phase images. When cells are touching, these halos can create a false cell boundary detected by the algorithm. Thus, the algorithm sometimes assigns the same pixels to neighboring cells leading to overlapping cell segmentations. (B-C) Example cells imaged with 40X (B) and 63X (C) objectives. Initial Segmentations: Overlaps between the initial segmentations of the neighboring cells are highlighted as white areas. Each cell segmentation is represented with a different color. Example Cell Score: Each individual cell has a cell score, which carries weights for whether a pixel should belong to the cell. Previous Algorithm: Overlapping regions among the initial segmentations were excluded from the segmentation in the previous algorithm [30]. Improved Segmentation: In the new algorithm such overlapping regions are distributed among the cells based on their scores, which significantly improves the segmentation at the cell boundaries. (D-E) Comparison of cell areas with and without distributing the overlapping regions for 40X (D) and 63X (E) objectives. Cells imaged over 10 hours (100 time points) were segmented with and without distributing the overlapping segmented regions. By distributing these intersections, the majority of cells gained cell area (75% for 40X and 97% for 63X. See Table 2.). Percent area gain is calculated by dividing the difference of the cell area with and without distributing the intersections by the area with distributing the intersections and then multiplying the result by 100. Next, the average percent cell area gain versus average size is plotted. To this end, cell sizes are grouped in 50-pixel increments (40X) or in 100-pixel increments (63X). The average size of each group is plotted against the average percent size gain in that group. The error bars show the standard error of the mean. Note that for small cells (buds) area gain percentage is higher than mother cells.

Fig 5. Segmentation of cells subject to varying levels of pheromone treatment.

(A-E) First column shows the phase images of cln1 cln2 cln3 cells without α-factor (A) and with varying levels of α-factor treatment (B-E). Note that the shapes get progressively more irregular as the concentration of the α-factor increases. Second column shows the histogram of percent area gain by distributing the overlapping segmentation regions. Note that histograms are capped at 10%. Third column shows the relationship between size of the cell and the percent cell area gain. The cell sizes are grouped in 100-pixel increments. The average size of each group is plotted against the average percent size gain in that group. The error bars show the standard error of the mean. Note that for small cells area gain percentage is higher than that for larger cells.

Fig 6. Quantification of the Erg6-TFP intensity at the cell periphery.

(A-B) Example cells imaged with 40X (A) and 63X (B) objectives. The cell segmentations with the previous algorithm (without distributing the overlapping initial segmentations, but by removing them) and with the new algorithm (with distributing the overlapping initial segmentations) are shown side-by-side. Note that the cells are the same cells as shown in Fig 4. (C-D) Comparison of the Erg6-TFP mean intensity at the cell periphery with and without distribution the overlaps for 40X (C) and 63X (D) objectives. The same cells as in Fig 4 are used for this quantification. Percent quantification difference is calculated by dividing the absolute value of the quantification difference by the quantification with distributing the overlaps and then multiplying by 100. Next, the average percent cell area gain versus the average percent quantification difference is plotted. To this end the cells are grouped in 4% cell area gain increments and the average percent quantification difference is plotted against the mean of each group. The error bars show the standard error of the mean.

Fig 7. Robustness of the segmentation algorithm.

Robustness to errors in the seed. (A) Example cell: The seed of the example cell is perturbed by randomly removing 40% of the seed. The algorithm uses this perturbed seed to segment the cell at time point p-1 and recovers the cell with only minor mistakes. The algorithm fully recovers the cell in two time points. Note that the algorithm segments the cells backwards in time, thus time points (i.e. frame numbers) are decreasing. (B) The seeds of 340 cells were perturbed by randomly removing 10–90% of the seed. The cells are grouped based on the severity of perturbation, i.e. percent seed area removed, in 25% increments. Mean fraction of fully recovered cells is plotted for each group. Note that out of 340 cells, only 9 of them were not recovered by the algorithm. (C) The cells are grouped based on the perturbation in 25% increments and the average number of time points required to fully recover the correct cell segmentation is plotted for each group. Number of time points required to fully recover the cells increase with the severity of the seed perturbation. The error bars show standard error of the mean. Robustness to time interval between frames. (D) Example colony used for the quantification presented in Table 4. The correct segmentation at time t is used as a seed to segment the images taken at t-24 min and t-60 min. All cells are segmented accurately when the time interval between the seed and the image is 24 minutes. However, when this interval is raised to 60 minutes, a major error is introduced (See the over-segmented cell in red and green.).

Fig 8. Utilizing a fluorescent channel for improving the segmentation of sporulating cells.

(A) Example phase image, GFP-channel image and the composite image. In the phase image, spores have very bright patches unlike cycling cells. The composite image is created using the phase and GFP-channel images. Note that Vma1-GFP channel is not dedicated to segmentation. (B) Segmentation results using the phase image and using the composite image. Using the composite image corrects for the slight out of focus phase image and significantly improves the segmentation. (C-D) Comparison of segmentations with phase and composite images. Example cells were imaged for 20 hours (100 time points) and segmented with phase or the composite images. (C) Out of 32868 cell segmentation events, 89.5% of them have a greater area when the composite image is used for segmentation. (D) Comparison of errors in segmentation with phase or composite images. Blue no error, green minor error. Minor errors decreased significantly when composite images were used for segmentation.

Fig 9. Segmentation of bright-field images.

(A) Example bright-field image. (B) Bright-field image is processed before segmentation by applying a top-hat transform to its complement. (C) Segmentation of the image. Each cell boundary is marked with a different color.

Fig 10. Overall performance of segmentation examples.

Sorted cell traces for every example case. Time points where the cell is not yet born are dark blue. Correct segmentations are labeled blue, minor errors green and major segmentation errors yellow. The errors were scored manually. For quantification see Table 7.

Fig 11. Overview of the segmentation and tracking algorithm.

First, the automated seeding step segments the image of the last time point. This seed is fed into the algorithm, which segments the images backwards in time and uses the segmentation of the previous time point as a seed for segmenting the next time point. The segmentation at a given time point is summarized in the blue box. Improvements over the previously published algorithm [30] are highlighted in red boxes.