Virus Detection and Identification in Minutes Using Single-Particle Imaging and Deep Learning

Abstract

The increasing frequency and magnitude of viral outbreaks in recent decades, epitomized by the COVID-19 pandemic, has resulted in an urgent need for rapid and sensitive diagnostic methods. Here, we present a methodology for virus detection and identification that uses a convolutional neural network to distinguish between microscopy images of fluorescently labeled intact particles of different viruses. Our assay achieves labeling, imaging, and virus identification in less than 5 min and does not require any lysis, purification, or amplification steps. The trained neural network was able to differentiate SARS-CoV-2 from negative clinical samples, as well as from other common respiratory pathogens such as influenza and seasonal human coronaviruses. We were also able to differentiate closely related strains of influenza, as well as SARS-CoV-2 variants. Additional and novel pathogens can easily be incorporated into the test through software updates, offering the potential to rapidly utilize the technology in future infectious disease outbreaks or pandemics. Single-particle imaging combined with deep learning therefore offers a promising alternative to traditional viral diagnostic and genomic sequencing methods and has the potential for significant impact.

Keywords: SARS-CoV-2; fluorescence microscopy; influenza; machine learning; viral diagnostics.

Figure 1

A fluorescent labeling and imaging strategy to detect viruses. (A) Overview: (i) Viruses were labeled and imaged. Individual signals were isolated, and a convolutional neural network (CNN) was trained to exploit differences in the features of different viruses to identify them. (ii) Signals from unknown samples can then be fed into the trained CNN to allow (iii) virus classification. (B) Representative fields of view (FOVs) of infectious bronchitis virus (CoV (IBV)). 1 ×10⁴ PFU/mL virus was labeled with 0.23 M SrCl₂, 1 nM Cy3 (green) DNA, and 1 nM Atto647N (red) DNA before being imaged. Green DNA was observed in the green channel (top panels) and red DNA in the red channel (middle panels); merged red and green localizations are shown in the lower panels. Scale bar, 10 μm. A negative control where DNA was replaced with water is included. (C, D) Zoomed-in images from (B); white boxes represent examples of colocalized particles. Scale bar, 5 μm. (E) Segmentation process: (i) Cropped FOV from the red channel. (ii) Intensity filtering applied to (i) to produce a binary image. (iii) Area filtering applied to (ii) to include only 10–100 pixel objects. (iv) Location image associated with (i). (v) Colocalized signals in the location image. (vi) Bounding boxes (BBXs) found from (iii) drawn onto (v). Non-colocalized objects (cyan) are rejected. (vii) Colocalized objects (red) are drawn over (i). Scale bar, 10 μm. (F) Plot of mean number of BBXs per FOV for labeled CoV (IBV) and the negative controls. Error bars represent the standard deviation of 81 FOVs from a single slide. Statistical significance was determined by one-way ANOVA, *P = 6.01 × 10^–22.

Figure 2

Design of a convolutional neural network to classify imaged viruses. (A) Representative FOVs of fluorescently labeled coronavirus (CoV (IBV)), two strains of H3N2 influenza (A/Udorn/72 (Udorn) and A/Aichi/68 (X31)), an H1N1 influenza strain (A/PR8/8/34 (PR8), and a negative control where virus was substituted with allantoic fluid. The samples were immobilized and labeled with 0.23 M SrCl₂, 1 nM Cy3 (green) DNA, and 1 nM Atto647N (red) DNA before being imaged. Merged red and green localizations are shown; examples of colocalizations are highlighted with white boxes. Scale bar, 10 μm. (B–D) Normalized frequency plots of the maximum pixel intensity, area, and semimajor-to-semiminor axis ratio within the BBXs of the four different viruses. Values taken from 81 FOVs from a single slide for each virus. Statistical significance was determined by one-way ANOVA, P values depicted above graphs. (E) Illustration of the 15-layer shallow convolutional neural network. Following the input layer (inputs comprising BBXs from the segmentation process), the network consists of three convolutions (stages 1–3). Stages 1 and 2 each contain a ReLU layer to introduce non-linearity, a batch normalization layer (not shown), and a max pooling layer, while stage 3 lacks a max-pooling layer. The classification stage has a fully connected layer and a softmax layer to convert the output of the previous layer to a normalized probability distribution, allowing the initial input to be classified.

Figure 3

Network validation results for laboratory-grown virus strains. (A) Network validation results shown as a confusion matrix: rows, predicted class (output class); columns, true class (target class); right column, positive and negative predictive values (percentages of BBXs that are correctly and incorrectly predicted); bottom row, sensitivity and specificity. (B) Confusion matrix of CoV (IBV) positive and negative samples. (C, D) Confusion matrices of CoV (IBV) vs influenza Udorn or PR8. (E) Confusion matrix of influenza PR8 vs influenza WSN. (F) Confusion matrix of CoV (IBV) vs a pooled dataset of the virus-negative control and three influenza A strains. (G) A trained network is robust over significant time. The network was trained on data from images of the virus IBV and allantoic fluid as a negative control. Each data point (orange for sensitivity; green for specificity) corresponds to the classification result for signals detected at different dates over a period of 135 days. Error bars represent standard deviation. (H) Defining the limit of detection for accurate machine learning classification. Increasing concentrations of IBV were labeled and imaged, the resulting images were fed into the trained network. The number of normalized positive particles (positive particles/all particles) increased linearly with increasing virus concentration. Error bars represent standard deviation. The limit of detection (LOD) was defined as 6 × 10⁴ PFU/mL, with 99.85% certainty.

Figure 4

A deep learning network can differentiate viruses in clinical samples. (A) Workflow for training and validation of clinical samples. Samples were collected from 33 patients, labeled, and imaged on a microscope over three different days. The images were processed to isolate the individual signals into BBXs. 70% of the BBXs were used to train a convolutional neural network (CNN), resulting in a trained model. The remaining 30% of the BBXs were used to validate the trained model, providing the result in a confusion matrix. (B) Confusion matrix showing that a trained network could differentiate positive (Alpha variant) SARS-CoV-2 and negative clinical samples. (C) Confusion matrix showing that a trained network could differentiate influenza A (Flu A) positive clinical samples from negative samples. (D) Confusion matrix showing that a trained network could differentiate SARS-CoV-2 samples (original Wuhan variant) from seasonal human coronavirus (hCoV) samples.

Figure 5

Independent testing of the trained network with clinical samples. (A) Schematic of workflow of independent testing. Previously unseen samples are imaged, the images are processed into BBXs which are fed through a trained network. When necessary, a chi-squared statistical test is performed to test the null hypothesis that the sample is negative. If the p-value is smaller than a pre-set confidence threshold, the null hypothesis is rejected and the sample is classified as positive. If the p-value is greater than the threshold, the sample is classified as negative. (B) Summary of independent testing results using multiple trained models. 51 patient samples that were not used for network training or validation were run through different trained versions of the network, detailed on the bottom of the plot. Some samples were tested in multiple versions of the network, for further details see Sup.Table 2. Chi-squared tests were carried out to classify the samples (see Sup.Figure 8 and Table 2; samples with a p-value smaller than a preset confidence threshold were classified as positive) and the results compared to RT-PCR. 50 out of 51 samples were classified correctly (incorrect classification shown in black), giving an overall sample accuracy of 98.03%. Results were obtained using two different microscopes (see Experimental Methods). (C) Summary of independent testing results using a single trained network. 104 patient samples were analyzed as in B), but tested in a single trained network (SARS-CoV-2 vs negative). (D) Variant classification of clinical samples identified as positive in (C). The BBXs from images of the clinical samples classified as positive by the first network were passed through a second trained model (Wuhan + Alpha SARS-CoV-2 vs Delta SARS-CoV-2).