TNTdetect.AI: A Deep Learning Model for Automated Detection and Counting of Tunneling Nanotubes in Microscopy Images

Abstract

Background: Tunneling nanotubes (TNTs) are cellular structures connecting cell membranes and mediating intercellular communication. TNTs are manually identified and counted by a trained investigator; however, this process is time-intensive. We therefore sought to develop an automated approach for quantitative analysis of TNTs.

Methods: We used a convolutional neural network (U-Net) deep learning model to segment phase contrast microscopy images of both cancer and non-cancer cells. Our method was composed of preprocessing and model development. We developed a new preprocessing method to label TNTs on a pixel-wise basis. Two sequential models were employed to detect TNTs. First, we identified the regions of images with TNTs by implementing a classification algorithm. Second, we fed parts of the image classified as TNT-containing into a modified U-Net model to estimate TNTs on a pixel-wise basis.

Results: The algorithm detected 49.9% of human expert-identified TNTs, counted TNTs, and calculated the number of TNTs per cell, or TNT-to-cell ratio (TCR); it detected TNTs that were not originally detected by the experts. The model had 0.41 precision, 0.26 recall, and 0.32 f-1 score on a test dataset. The predicted and true TCRs were not significantly different across the training and test datasets (p = 0.78).

Conclusions: Our automated approach labeled and detected TNTs and cells imaged in culture, resulting in comparable TCRs to those determined by human experts. Future studies will aim to improve on the accuracy, precision, and recall of the algorithm.

Keywords: TNT; artificial intelligence; automated cell counting; biomarker; cancer; cells; deep learning; machine learning; microscopy; tunneling nanotubes.

Figure 1

(A) Two TNTs that were successfully captured by the deep learning model (true positives). (B) The image from (A) is enhanced for improved TNT visibility. (C) A TNT-appearing structure that was mistakenly identified as a TNT by the model (false positive). Images (B,C) were generated with Fiji software and were adjusted for their brightness and contrast by setting minimum and maximum displayed value to 20 and 100, respectively, for improved visibility of the structures (this image modification is not necessary for the deep learning model to work).

Figure 2

(A) Tiled image with shadows at edges of the tiles and (B) the same image with the shadows removed to prevent a high false-positive detection rate.

Figure 3

Flow diagram of AI-based TNT detection. Images were (Step 1) pre-processed for label correction and (Step 2) subdivided into a matrix of smaller image regions (‘patches’) that were classified as either containing or not containing any TNT structures, and pixel-wise classified regarding whether each pixel belonged to a TNT structure or not (see Supplementary Figures S1 and S2 and Supplementary Table S2). In (Step 3), the numbers of TNTs and cells were counted, and the TNT-to-cell ratio (TCR) was calculated (each colored object is an individual cell) and confusion matrix was reported (see Table 2 and Supplementary Table S3). XOR = bitwise exclusive or operator.

Figure 4

(A) Original image containing large pockets of TNT-free spaces. (B) After correcting edge artefacts as shown in Figure 2, the TNT-containing “patches” (yellow squares) showed where TNTs were captured within the matrix of smaller image regions. See Supplementary Figures S1 and S2.

Figure 5

TNTs detected from two cropped images. (A,E) are the cropped raw images. (B,F) are the manually marked labels. (C,G) are the heatmap versions after prediction. (D,H) are the predicted TNTs.