Clinical evaluation of commercial automated SARS-CoV-2 immunoassays

Abstract

Objective: Numerous immunoassays for detecting antibodies directed against SARS-CoV-2 have been rapidly developed and released. Validations of these have been performed with a limited number of samples. The lack of standardisation might lead to significantly different results. This study compared ten automated assays from six vendors in terms of sensitivity, specificity and reproducibility.

Methods: This study compared ten fully automated immunoassays from the following vendors: Diasorin, Epitope Diagnostics, Euroimmun, Roche, YHLO, and Snibe. The retrospective part of the study included patients with a laboratory-confirmed COVID-19 infection, and controls comprised patients with a suspected infection, in whom the disease was excluded. Furthermore, biobanked sera were taken as negative controls (n = 97). The retrospective part involved four groups: (1) laboratory-confirmed COVID-19 infection (n = 183); (1B) suspected COVID-19 infection (n = 167) without a qRT-PCR result but positive serological results from at least two different assays, and suspected COVID-19 infection due to a positive serological result from the Roche assay (n = 295); (2) biobanked sera obtained from patients before the emergence of SARS-CoV-2 (n = 97) as negative controls; and (2A) probably COVID-19-negative sera with negative serological results from at least two different assays (n = 152).

Results: Overall diagnostic sensitivities were: Euroimmun (IgA) 87%; Epitope Diagnostics (IgG) 83%; YHLO (IgG) 77%; Roche (IgM/IgG) 77%; Euroimmun (IgG) 75%; Diasorin (IgG) 53%; Epitope Diagnostics (IgM) 52%; Snibe (IgG) 47%; YHLO (IgM) 35%; and Snibe (IgM) 26%. Diagnostic specificities were: YHLO (IgG) 100%; Roche, 100%; Snibe (IgM/IgG) 100%; Diasorin (IgG) 97%; Euroimmun (IgG) 94%; YHLO (IgM) 94%; Euroimmun (IgA) 83%.

Conclusion: Assays from different vendors substantially varied in terms of their performance. These findings might facilitate selection of appropriate serological assays.

Keywords: Automated SARS-CoV-2 antibody detection; High throughpu testing; Longitudional monitoring of antibody development; Pandemic control; Seroconversion; Serpprevalence.

Figure 1

Precision box-whiskers plot of the inter-assay and intra-assay variations in the IgM and IgG SARS-CoV-2 assay from Shenzhen YHLO Biotech and Roche Elecsys Anti-SARS-CoV-2 ECLIA.

Figure 2

Z-Score comparison. Comparison between the IgM and IgG SARS-CoV-2 assay from Shenzhen YHLO Biotech and the IgM/IgG assay by Roche Diagnostics. Data were transformed to Z-scores, as described in the materials and methods section, and the cut-off values were uniformly set to log (1). Measurement of qRT-PCR-confirmed COVID-19 cases (n = 59) and pre-pandemic negative controls (n = 50).

Figure 3

Diagnostic sensitivity. Comparison of serological results for all immunoassays listed in Table 2: Serum specimens of qRT-PCR-confirmed patients (Class 1) were analysed. Blood from patients with infection was drawn on various days after positive qRT-PCR results, as shown in the figure. The dates on which the qRT-PCRs were performed are unknown for 13 samples.

Figure 4

AUROC curves. AUROC graphs for the highest ((A) Shenzhen YHLO Biotech IgG, AUC: 0.97) and lowest ((B) Snibe IgM, AUC: 0.66) values; detailed information can be found in Table 3.

Figure 5

Longitudinal monitoring. Kinetics of antibody concentrations were analysed for 15 patients with qRT-PCR-confirmed COVID-19 infections. Multiple blood samplings were performed for up to 14 weeks after qRT-PCR. The cut-off values for the Shenzhen YHLO Biotech assays (10 AU/mL) are indicated with horizontal lines. The courses of antibody concentrations were grouped into four categories: (I) decreasing (blue), (II) increasing (orange), (III) steady (green), and (IV) variable (yellow). Most interestingly, antibody formation could not be detected for IgM or IgG in one case, even 9 weeks after a positive qRT-PCR result (triangle symbols, Patient 1).