Computer vision meets microfluidics: a label-free method for high-throughput cell analysis

Abstract

In this paper, we review the integration of microfluidic chips and computer vision, which has great potential to advance research in the life sciences and biology, particularly in the analysis of cell imaging data. Microfluidic chips enable the generation of large amounts of visual data at the single-cell level, while computer vision techniques can rapidly process and analyze these data to extract valuable information about cellular health and function. One of the key advantages of this integrative approach is that it allows for noninvasive and low-damage cellular characterization, which is important for studying delicate or fragile microbial cells. The use of microfluidic chips provides a highly controlled environment for cell growth and manipulation, minimizes experimental variability and improves the accuracy of data analysis. Computer vision can be used to recognize and analyze target species within heterogeneous microbial populations, which is important for understanding the physiological status of cells in complex biological systems. As hardware and artificial intelligence algorithms continue to improve, computer vision is expected to become an increasingly powerful tool for in situ cell analysis. The use of microelectromechanical devices in combination with microfluidic chips and computer vision could enable the development of label-free, automatic, low-cost, and fast cellular information recognition and the high-throughput analysis of cellular responses to different compounds, for broad applications in fields such as drug discovery, diagnostics, and personalized medicine.

Keywords: Electrical and electronic engineering; Optical sensors.

Fig. 1

Cell recognition through the human eyes and brain and computer vision. PIT: posterior IT, CIT: central IT, AIT: anterior IT

Fig. 2

Computer vision for image analysis. a Four major applications in the field of computer vision. (i) input an image and determine the class to which the image belongs; (ii) detect all objects in an image, determine their classes and locate the positions of the objects; (iii) on the basis of object detection, semantic segmentation requires further determination of which pixels belong to a class in an image; (iv) determine which pixels each object contains on the basis of object detection; b Convolutional neural network for cell feature extraction and classification. CNN can detect the features of corners, edges, and lines in the first layers, and its middle layer represents a combination of these low-level features, while at deeper levels, there are features that give the most weight to the cells that need to be separated, such as the membrane structure and dot patterns^,

Fig. 3

Traditional computer vision algorithm-assisted microfluidic cytometry. a The three main types of deformability cytometry. (i) cDC measured the time needed for cells to pass through the constricted channel; (ii) sDC and (iii) xDC use hydrodynamic flow to induce cell deformation without direct contact and infer cell deformability from the image-based evaluation of cell shape. b Nine common shape descriptors. They are used to measure the degree of deformation of cells after being focused by the buffer. Different features are selected to construct a scatter plot that shows the ranges of features of different cell classes. c Dynamic tracking of single cells in a microfluidic channel. Multiple ROIs are set up in the channels to collect time series images of cellular deformation

Fig. 4

On-chip cell detection and classification with computer vision technologies. a CV-based image-activated microfluidic cell sorting. Image segmentation algorithms and object detection neural networks in computer vision were applied to quickly collect cell images in a high-throughput manner and output the sorting decision to the downstream piezoelectric actuator. b The YOLOv4 algorithm with multiframe image correlation to detect CTCs in a sheathless and label-free manner. Multiple frames of a cell in the flow channel can reflect the phenotype of that cell from multiple perspectives, and together, these images are used to make classification decisions, greatly improving the accuracy of cell classification as well as the robustness of the model. c Lensless portable flow cytometer for natural water analysis. This device uses a partially coherent lensless holographic microscope to capture images with diffraction patterns of flowing objects in microchannels and then converts these diffraction patterns into chromatic variations for a deep learning neural network to be trained for the classification and identification of algae

Fig. 5

CV-based drug screening system. a Prediction of the therapeutic effect of drugs on tumor spheres by a convolutional neural network. The chip can simultaneously test six drug conditions and capture bright-field images and live/dead cell staining images of tumor cells after drug treatment. The bright-field images are input into a convolutional neural network to predict the viability of cells, and the living/dead cell staining images are used as the true values to evaluate the accuracy of the prediction results. b Combination of a hydrogel droplet platform and computer vision to screen the antisolvent crystallization conditions of active pharmaceutical ingredients. The method collects images of hydrogel droplets containing different drug crystals in serpentine channels and detects different drug crystal shapes in the hydrogel droplets by using an object detection algorithm. c Real-time drug screening by ultralarge-scale high-resolution imaging and computer vision. Video clips of Ca2+ ion signals in cells are recorded at 30 Hz and then analyzed offline. Within each region of interest, all image frames are accumulated to synthesize a grayscale map, and individual cells are then identified by the binarization algorithm ImageJ. The fluorescence intensity of these cells in each frame are extracted and resolved, revealing the rhythmic response of cardiomyocytes following drug injection

Fig. 6

CV-assisted point-of-care testing devices. a Herringbone microfluidics for multiple exosome detection. The detection of exosomes can be simplified to read the photonic signals of barcodes with the imaging of charge-coupled-device and the digital image processing methods to achieve higher sensitivity and accuracy. b A mobile device supported by machine learning for high-precision pH classification. The color values from individual components are extracted by multiple traditional image processing techniques, such as gray/HSV conversion, histogram equalization and blurring, and calculating the difference in the vector space can improve the precision of pH classification. c A low-cost system for high-precision detection of C. elegans. This system utilizes a Mask R-CNN to automatically detect C. elegans in an end-to-end manner, which yields reliable and precise location information. The neural network structure has been optimized and supported by hardware and software platforms, making it easy to deploy on mobile devices for a wide range of bioanalytical tasks