Cellular behavior analysis from live-cell imaging of TCR T cell-cancer cell interactions

Abstract

T cell therapies, such as chimeric antigen receptor (CAR) T cells and T cell receptor (TCR) T cells, are a growing class of anti-cancer treatments. However, expansion to novel indications and beyond last-line treatment requires engineering cells' dynamic population behaviors. Here we develop the tools for cellular behavior analysis of T cells from live-cell imaging, a common and inexpensive experimental setup used to evaluate engineered T cells. We first develop a state-of-the-art segmentation and tracking pipeline, Caliban, based on human-in-the-loop deep learning. We then build the Occident pipeline to collect a catalog of phenotypes that characterize cell populations, morphology, movement, and interactions in co-cultures of modified T cells and antigen-presenting tumor cells. We use Caliban and Occident to interrogate how interactions between T cells and cancer cells differ when beneficial knock-outs of RASA2 and CUL5 are introduced into TCR T cells. We apply spatiotemporal models to quantify T cell recruitment and proliferation after interactions with cancer cells. We discover that, compared to a safe harbor knockout control, RASA2 knockout T cells have longer interaction times with cancer cells leading to greater T cell activation and killing efficacy, while CUL5 knockout T cells have increased proliferation rates leading to greater numbers of T cells for hunting. Together, segmentation and tracking from Caliban and phenotype quantification from Occident enable cellular behavior analysis to better engineer T cell therapies for improved cancer treatment.

Figure 1.. Live-cell imaging serves as a platform for cellular behavior analysis.

(a) Traditional analysis of immunotherapy and tumor co-cultures monitors the fluorescence of a cancer nuclear marker over time to quantify killing rates, missing dynamic behaviors of T cells and tumor cells visible in brightfield images. (b) With novel machine learning methods for segmentation, tracking, and spatiotemporal modeling, brightfield images can be used to phenotype cellular behavior, particularly interaction dynamics between T cells and cancer cells.

Figure 2.. A deep learning approach to cell segmentation and tracking using Caliban.

(a) Caliban takes a movie of fluorescently-labeled nuclei as input and then generates a nuclear segmentation mask for each frame. Features for each cell in a frame are extracted and passed through a neighborhood encoder model to generate a vector embedding for each cell. These embeddings and cell positions are passed into the tracking inference model, which predicts the probability that each pair of cells between frames is the same, is different, or has a parent–child relationship. These probabilities are used as weights for linear assignment to construct cell lineages on a frame-by-frame basis. (b) The neighborhood encoding model takes as input an image of each cell, its centroid position, and three metrics of morphology (area, perimeter, and eccentricity). A vector embedding of each input is used as node weights in a graph attention network^,, where edges are assigned to cells within 64 pixels (41.6 μm) of each other. The final neighborhood embedding for each cell captures the appearance of that cell and its spatial relationship with its neighbors in that frame. (c) The tracking inference model performs pairwise predictions on cells in frame $t_n$ to cells in frame $t_{n+1}$. The model is given neighborhood embeddings and centroid positions of cells in the previous seven frames $\left[t_{n-7},t_n\right]$ to compare with cells in frame $t_{n+1}$. The temporal context of the previous seven frames is modeled using long short-term memory (LSTM) layers. Ultimately, the model outputs a set of effective probabilities ($p_{\text{same}}$, $p_{\text{diff}}$, and $p_{\text{parent-child}}$) for each pair of cells between frame $t_n$ and frame $t_{n+1}$. (d) The performance of Caliban and that of four other tracking methods were evaluated on the test split of DynamicNuclearNet. Tracking performance on ground-truth segmentations is excluded for EmbedTrack because it is an end-to-end method that generates segmentations as a part of tracking. TRA: tracking accuracy in the Cell Tracking Challenge. (e) A sample montage from DynamicNuclearNet with predictions from Caliban. Circles highlight the correct identification of three division events. (Scale bars = 26 μm)

Figure 3.. Extracting a catalog of cellular phenotypes with Occident.

(a) Morphology is an indicator of cell state. T cells that are interacting with cancer cells change from their round shape to attack cancer cells while cancer cells aggregate together under distress. Segmentation masks are able to capture putative cells and cell agregates for both cell types. (b) Increased T cell expansion with beneficial knockouts. (c) Cancer cell population from segmentation over time reveals greater killing in wells with beneficial T cell perturbations. Fluorescence of the cancer nuclear marker is a lagging indicator of cancer cell population. (d) Cancer cells in RASA2 KO wells have lower average area, indicating greater stress from T cell attack. (e) SH KO and CUL5 KO T cells are smaller when attached to cancer cells, while RASA2 KO T cells increase in area when attached, likely due to greater activation. (f) The ratio of mean cancer cell clump area to mean individual cancer cell area increases over time as stressed cancer cells aggregate. (g) T cells with beneficial genetic knockouts become less round over time, indicating greater anti-cancer activity. (h) Cancer cell division events decreased in wells with T cells with beneficial genetic knockouts. (i) Increased local T cell density decreases the probability of cancer cell division and growth. T cells with beneficial genetic knockouts have a greater inhibitory effect on cancer cell division. (j) RASA2 KO T cells substantially decreased the average speed of cancer cells, suggesting greater anti-cancer effect during interactions. (k) T cells with beneficial genetic knockouts had higher average speed than control T cells, suggesting greater activity.

Figure 4.. Occident characterizes the behavior of T cells and cancer cell aggregates during cell-cell interactions.

(a) An example interaction between a SH KO T cell and a cancer cell, and the same cells 4 minutes later. (b) Wells with RASA2 KO T cells have a higher average number of cancer cell – T cell interactions across time than CUL5 KO and SH KO wells. (c) Average T cell size increased after interaction across conditions. (d), T cell roundness decreased as they interact with cancer cells. (e) Cancer aggregate speed decreased during interactions with T cells. (k) Cancer cells decreased in area after interaction with T cells.

Figure 5.. An interpretable spatiotemporal model of cell organization characterizes interaction with cancer cell duration, recruitment, and proliferation in T cells

a, Schematic of Markov transition model and identification of proliferation and recruitment events. b, Estimated Markov transition probabilities of states at $t+1$ from 1 T cell, 1 cancer cell at time $t$ by genetic perturbation c, difference in Markov transition probabilities at $t+1$ from 1 T cell, 1 cancer cell at time $t$ from SH KO control. d Mean time of interaction estimated from tracks and Markov transition probabilities, controlling for two frame (eight minute) minimum cutoff. e, Probability of transition from 1 T cell, 1 cancer cell at time $t$ to 2 T cells, 1 cancer cell at $t+1$ from recruitment and from proliferation, by genetic perturbation.