Siamese deep learning video flow cytometry for automatic and label-free clinical cervical cancer cell analysis

Abstract

Automatic and label-free screening methods may help to reduce cervical cancer mortality rates, especially in developing regions. The latest advances of deep learning in the biomedical optics field provide a more automatic approach to solving clinical dilemmas. However, existing deep learning methods face challenges, such as the requirement of manually annotated training sets for clinical sample analysis. Here, we develop Siamese deep learning video flow cytometry for the analysis of clinical cervical cancer cell samples in a smear-free manner. High-content light scattering images of label-free single cells are obtained via the video flow cytometer. Siamese deep learning, a self-supervised method, is built to introduce cell lineage cells into an analysis of clinical cells, which utilizes generated similarity metrics as label annotations for clinical cells. Compared with other deep learning methods, Siamese deep learning achieves a higher accuracy of up to 87.11%, with about 5.62% improvement for label-free clinical cervical cancer cell classification. The Siamese deep learning video flow cytometry demonstrated here is promising for automatic, label-free analysis of many types of cells from clinical samples without cell smears.

Fig. 1.

The schematic diagram of Siamese deep learning video flow cytometry. A laser beam with a 532 nm excitation wavelength is used to illuminate the excitation area of the flow chamber after the objective. Cell lineage cells and clinical samples are processed into single-cell suspension as sample stream driven by the syringe pump. Single-cell sequence is obtained by hydrodynamic focusing. The 2D light scattering patterns are obtained in a direction perpendicular to the excitation light beam with a high-speed CMOS sensor. Acquired data is analyzed by Siamese deep learning algorithm.

Fig. 2.

The framework of Siamese deep learning. (a) is the whole process and structure of Siamese deep learning, which makes up of the pretext task and the downstream task. In the pretext task, cultured samples (contrast data) and clinical caner samples form image pairs, which are fed into the Siamese network to obtain the similarity metric. Processed dataset is obtained after excluding some interferences, which is based on the score ranking of similarity metrics. On this basis, downstream task enables further classification and analysis of clinical samples. (b) shows the Siamese network structure. Siamese network has two sub-networks with the same structure and shares the same training parameters. Paired images are processed by the network to obtain the similarity metric. (c) gives the layers of Inception V3 network and (d) is the modules layers used in Inception V3 networks.

Fig. 3.

Representative experimental 2D light scattering images. (a) Representative FOIs for the 2D light scattering patterns of single cells from cell lineage cells. (b) Representative FOIs from clinical cells.

Fig. 4.

Classification results of cultured cell lineage cells and clinical cells with conventional deep learning. (a) The hot map gives the confusion matrix of classification. Caski and C33-A cells are identified with an accuracy of 92.07% and 88.72%, corresponding to an overall accuracy of 90.37%. (b) gives the mean accuracy for the classification of all the prediction set and clinical cells from normal and cancerous patients displayed by category. The prediction set is obtained from 13 clinical cases (4 cancer cases and 9 normal cases). To minimize random errors, three independent replicate tests of the classification are used.

Fig. 5.

Verification of Siamese network with cell lineage cells. (a) The workflow of verification process. Siamese network has the same sub-network structure and its input is image pairs. Patterns from two subtypes of cell lineage cells are randomly selected to make image pairs. The contrasting data are from Caski cells, where similar / dissimilar image pairs consist of the rest Caski cells or C33-A cells with selected contrasting cells, respectively. 400 image pairs are used for model training (training set and validation set ratio is 8:2, automatic random assignment), while 300 image pairs are for testing. (b) The result of predict set. The accuracy of similar pairs (Caski to Caski) and dissimilar pairs (C33-A to Caski) are 98% and 97%, respectively.

Fig. 6.

Building the initial image pairs of the clinical and cultured samples for Siamese deep learning. The points in the feature space of granularity are shown here. Gran 1, 2, and 3 (a.u) is the feature of the mean area, the minimum area and the number of speckles, respectively. Shaded blocks represent the main distribution area of the cells. The orange area represents the distribution of the contrastive data (C33-A) and the blue area is clinical cancer cells. Closer distance indicates greater similarity. In the blue area, some points intersect with (red arrow region) or away from (yellow arrow region) the contrastive data, in which points are randomly selected to make similar or dissimilar pairs with contrastive points in the orange area, respectively.

Fig. 7.

Comparison results of Siamese self-supervised learning and deep learning methods. The mean accuracy of clinical samples (13 cases of prediction set). Each point represents the mean of three random classifications. The averaged accuracy of Siamese deep learning is 87.11% ± 4.67%. Compared with the 81.49% ± 5.95% of supervised deep learning method, an improvement about 5.62% is achieved.