Machine learning enhanced cell tracking

Abstract

Quantifying cell biology in space and time requires computational methods to detect cells, measure their properties, and assemble these into meaningful trajectories. In this aspect, machine learning (ML) is having a transformational effect on bioimage analysis, now enabling robust cell detection in multidimensional image data. However, the task of cell tracking, or constructing accurate multi-generational lineages from imaging data, remains an open challenge. Most cell tracking algorithms are largely based on our prior knowledge of cell behaviors, and as such, are difficult to generalize to new and unseen cell types or datasets. Here, we propose that ML provides the framework to learn aspects of cell behavior using cell tracking as the task to be learned. We suggest that advances in representation learning, cell tracking datasets, metrics, and methods for constructing and evaluating tracking solutions can all form part of an end-to-end ML-enhanced pipeline. These developments will lead the way to new computational methods that can be used to understand complex, time-evolving biological systems.

Keywords: bioimage analysis; cell tracking; computer vision; machine learning (ML); optimisation; tracking.

FIGURE 1

Examples of automated cell tracking and lineaging and evaluation using btrack. (A) Tracking MDCK cells in culture (image data from Ulicna et al. (Ulicna et al., 2021)) (B) Tracking cells in C. elegans early embryo development (image data from Murray et al. (Murray et al., 2006)). (C) Tracking cells in D. melanogaster embryo development (image data from Amat et al. (Amat et al., 2014)). (D) Example btrack (Ulicna et al., 2021) lineage output, using default tracking parameters, on the C. elegans dataset from Murray et al. (Murray et al., 2006). The manually annotated ground truth tree is shown for reference. The propagation of a single tracking error, highlighted as a red arrow is shown, demonstrating the complexity of the tracking and lineaging problem. (E) Examples of typical errors in automated cell tracking. Vertices are denoted as circles, and correct edges are shown as bold black arrows, errors as red arrows and dashed arrows indicate the ground truth where errors have occurred.

FIGURE 2

Tracking, graphs and discrete optimization. (A) A sequence of image volumes showing a single mitotic branching event highlighted with white circles representing the vertices and dashed lines representing the ground truth edges. (B) A simplified directed hypothesis graph of the mitotic event. Each vertex represents a unique cell detection. Edge weights (black and red arrows) and hyperedge weights (blue arrows) are calculated as the posterior probability of linking and branching hypotheses given the evidence, and calculated using btrack. (C) There are several hypotheses to account for the appearance of new vertices. In hypotheses 1 & 2, edges link vertex 28 and either 29 or 267 respectively. In hypothesis 3, a hyperedge links vertex 28 to both 29 and 267, representing a mitotic event. (D) A simplified ILP optimization problem using the graphical model, and possible solutions. The A matrix is sparse with non-zero elements colored by hypothesis type (red - edge, blue - hyperedge, grey - terminus). The rows represent individual hypotheses and the columns are the vertex IDs forming part of the hypotheses. The optimal solution (which maximizes ρ ^⊤ x s.t. $\mathbf{A}\mathbf{x}=\vec{1}$ ) is highlighted with an asterisk.