Video frame prediction of microbial growth with a recurrent neural network

Abstract

The recent explosion of interest and advances in machine learning technologies has opened the door to new analytical capabilities in microbiology. Using experimental data such as images or videos, machine learning, in particular deep learning with neural networks, can be harnessed to provide insights and predictions for microbial populations. This paper presents such an application in which a Recurrent Neural Network (RNN) was used to perform prediction of microbial growth for a population of two Pseudomonas aeruginosa mutants. The RNN was trained on videos that were acquired previously using fluorescence microscopy and microfluidics. Of the 20 frames that make up each video, 10 were used as inputs to the network which outputs a prediction for the next 10 frames of the video. The accuracy of the network was evaluated by comparing the predicted frames to the original frames, as well as population curves and the number and size of individual colonies extracted from these frames. Overall, the growth predictions are found to be accurate in metrics such as image comparison, colony size, and total population. Yet, limitations exist due to the scarcity of available and comparable data in the literature, indicating a need for more studies. Both the successes and challenges of our approach are discussed.

Keywords: Recurrent Neural Network; deep learning; machine learning; microbial growth; population growth; video frame prediction.

Figure 1

Brightened snapshots at time steps 1, 4, 7, 10, and 13 for one of the videos obtained in Timm et al. (2017) for the 30 μm well. The two mutant strains of P. aeruginosa, T6SS-positive and T6SS-negative, appear in green and red, respectively.

Figure 2

Flow chart of the data and model pipeline including preprocessing steps, dataset separation, and validation metrics.

Figure 3

Convergence metrics of predRNN (Wang et al., 2021) during training. (A) MSE loss for training images. (B) MSE loss for validation images. (C) LPIPS loss for validation images.

Figure 4

Qualitative comparison between the groundtruth and predicted frames for four wells in the test dataset. In each figure, the upper panel represents the groundtruth frames, and the lower the predicted frames. The images progress from left to right and the wells are numbered according to their position in the test dataset. Data taken from Timm et al. (2017) then expanded in size and interpolated in time as explained in Section 2.

Figure 5

Quantitative comparison between the groundtruth and the predicted frames for the test wells in Figure 1. (A) Average mean-squared error (MSE) for all test wells; (B) Average learned perceptual image patch similarity (LPIPS) for all test wells. Each dot represents a frame.

Figure 6

Comparisons between the population curve extracted from the groundtruth frames and the corresponding predicted population curves. See text for an explanation of how the population curves were extracted.

Figure 7

Comparison of the number and size of individual colonies in the groundtruth and predicted frames for wells in the test dataset. The wells are numbered according to their position in the test dataset.