Label-free SARS-CoV-2 detection and classification using phase imaging with computational specificity

Abstract

Efforts to mitigate the COVID-19 crisis revealed that fast, accurate, and scalable testing is crucial for curbing the current impact and that of future pandemics. We propose an optical method for directly imaging unlabeled viral particles and using deep learning for detection and classification. An ultrasensitive interferometric method was used to image four virus types with nanoscale optical path-length sensitivity. Pairing these data with fluorescence images for ground truth, we trained semantic segmentation models based on U-Net, a particular type of convolutional neural network. The trained network was applied to classify the viruses from the interferometric images only, containing simultaneously SARS-CoV-2, H1N1 (influenza-A virus), HAdV (adenovirus), and ZIKV (Zika virus). Remarkably, due to the nanoscale sensitivity in the input data, the neural network was able to identify SARS-CoV-2 vs. the other viruses with 96% accuracy. The inference time for each image is 60 ms, on a common graphic-processing unit. This approach of directly imaging unlabeled viral particles may provide an extremely fast test, of less than a minute per patient. As the imaging instrument operates on regular glass slides, we envision this method as potentially testing on patient breath condensates. The necessary high throughput can be achieved by translating concepts from digital pathology, where a microscope can scan hundreds of slides automatically.

Fig. 1. Virus-particle classification using SLIM and machine learning.

a Sample-preparation protocol, viruses were deactivated, stained with Rhodamine B isothiocyanate, and dialyzed for two days to reduce fluorescence background, and then placed on a slide, fixed with 90% EtOH, and air-dried. b We added a SLIM module to a traditional phase-contrast microscope for quantitative-phase information. c SLIM and fluorescence were registered, single 48 × 48 spots were cropped from the image and segmented to provide a label for multiclass classification. d We synthesized a new dataset by randomly placing the cropped virus particles onto a background image acquired during the same experiment. A deep neural network was trained with this dataset to perform virus-particle classification. Given a SLIM image, the model will output a class label for each pixel in the image

Fig. 2. SLIM.

a Optical configuration of SLIM. b Image reconstruction, with color bar representing optical path length (s), in nm. c Profile through yellow dotted line in b, showing high sensitivity of SLIM over phase contrast

Fig. 3. Correlated SLIM-fluorescence imaging results for SARS-CoV-2.

a SLIM, colorbar represents optical path-length fluctuations in nm, and b fluorescence image, colorbar represents intensity in a.u., for the same field of view. c, d Cropped SLIM and fluorescence images from the region inside the white rectangle in a and b, yellow boxes highlight the correspondence between SLIM and fluorescence. e One 48 × 48 cropped image of SLIM, f fluorescence, and g corresponding segmentation mask prepared for AI. Another cropped set for h SLIM, i fluorescence, and j segmentation mask. Scale bar represents 5μm for a, b and 1 μm for e–j

Fig. 4. 3D tomograms.

Volume reconstruction of a SARS-CoV-2. b H1N1. c HAdV. d ZIKV. Surface reconstructions of e SARS-CoV-2. f H1N1. g HAdV. h ZIKV. All reconstructions were performed using the Amira software. Scalebars are representative of lateral dimensions of the respective particles

Fig. 5. Training a deep neural network to perform classification of virus particles for the second dataset.

a We used a modified version of U-Net for this semantic-segmentation task. Besides reducing the number of parameters in the network to around 0.8 million, we also added in residual connection and batch normalization for faster convergence. Model inference on images from the test set. b Synthesized images of mixed virus particles. c Ground truth label. d Model inference

Fig. 6. Model performance on the test dataset.

a The receiver-operating characteristic (ROC) curve of the model on the test dataset. The model achieved over 0.9 area-under-curve (AUC) for all four virus types on the test dataset. The area-under-curve (AUC) for each class is computed by setting that class as label 1 and all other classes (the three remaining virus types) as label 0. b The confusion matrix of the model inference on the test dataset. Each row represents the ground-truth label, while each column represents the prediction. For visualization purposes, each entry in the confusion matrix was normalized with respect to the number of true labels (sum of each row). The precision and recall are averaged across all images in the test dataset. Both the ROC curve and the confusion matrix are evaluated on a per-particle level, where weighted average is computed to resolve conflict in model raw prediction