Evaluation of commercially available fully automated and ELISA-based assays for detecting anti-SARS-CoV-2 neutralizing antibodies

Abstract

Rapid and accurate measurement of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV2)-specific neutralizing antibodies (nAbs) is paramount for monitoring immunity in infected and vaccinated subjects. The current gold standard relies on pseudovirus neutralization tests which require sophisticated skills and facilities. Alternatively, recent competitive immunoassays measuring anti-SARS-CoV-2 nAbs are proposed as a quick and commercially available surrogate virus neutralization test (sVNT). Here, we report the performance evaluation of three sVNTs, including two ELISA-based assays and an automated bead-based immunoassay for detecting nAbs against SARS-CoV-2. The performance of three sVNTs, including GenScript cPass, Dynamiker, and Mindray NTAb was assessed in samples collected from SARS-CoV-2 infected patients (n = 160), COVID-19 vaccinated individuals (n = 163), and pre-pandemic controls (n = 70). Samples were collected from infected patients and vaccinated individuals 2-24 weeks after symptoms onset or second dose administration. Correlation analysis with pseudovirus neutralization test (pVNT) and immunoassays detecting anti-SARS-CoV-2 binding antibodies was performed. Receiver operating characteristic (ROC) curve analysis was generated to assess the optimal threshold for detecting nAbs by each assay. All three sVNTs showed an excellent performance in terms of specificity (100%) and sensitivity (100%, 97.0%, and 97.1% for GenScript, Dynamiker, and Mindray, respectively) in samples collected from vaccinated subjects. GenScript demonstrated the strongest correlation with pVNT (r = 0.743, R² = 0.552), followed by Mindray (r = 0.718, R² = 0.515) and Dynamiker (r = 0.608, R² = 0.369). Correlation with anti-SARS-CoV-2 binding antibodies was variable, but the strongest correlations were observed between anti-RBD IgG antibodies and Mindray (r = 0.952, R² = 0.907). ROC curve analyses demonstrated excellent performance for all three sVNT assays in both groups, with an AUC ranging between 0.99 and 1.0 (p < 0.0001). Also, it was shown that the manufacturer's recommended cutoff values could be modified based on the tested cohort without significantly affecting the sVNT performance. The sVNT provides a rapid, low-cost, and scalable alternative to conventional neutralization assays for measuring and expanding nAbs testing across various research and clinical settings. Also, it could aid in evaluating actual protective immunity at the population level and assessing vaccine effectiveness to lay a foundation for boosters' requirements.

Figure 1

Graphical illustration for the principle of sVNT in comparison to pVNT. (A) Principle of ELISA-based surrogate virus neutralization test (sVNT) where anti-SARS-CoV-2 nAbs block the binding of HRP-conjugated RBD protein to the precoated hACE2 protein on the ELISA plate. (B) Principle of Mindray competitive binding NTAb immunoassay where anti-SARS-CoV-2 nAbs compete with the ACE2-ALP conjugate for RBD-binding sites on the magnetic beads. (C) Mechanism of pseudovirus neutralization test (pVNT) where anti-SARS-CoV-2 nAbs block the binding of SARS-CoV-2 spike (S) protein to human ACE2 receptor on the host cell surface. All illustrations were created using BioRender. Figures (A) and (C) were adapted from Wang et al..

Figure 2

Dot plot of nAb results obtained from (A) GenScript SARS-CoV-2 surrogate virus neutralization test (sVNT), (B) Dynamiker sVNT, (C) Mindray NTAb sVNT, and (D) pseudovirus neutralization test (pVNT), using samples collected from healthy controls, SARS-CoV-2 infected patients, and COVID-19 vaccinated individuals. Individual points obtained, median values, and interquartile ranges (IQR) are shown. The dotted lines represent the cutoff at 30% inhibition for GenScript's and Dynamiker's sVNT, 33.1 IU/ml for Mindray's sVNT, and 20% neutralization for pVNT. P values were calculated using one-way analysis of variance (ANOVA) test. *p < 0.05, ***p < 0.001, **** p < 0.0001.

Figure 3

Correlation and linear regression analysis between each sVNT and serological assays detecting anti-SARS-CoV-2 binding antibodies, including anti-S and N IgG antibodies (orange), anti-RBD IgG (violet), and anti-RBD total antibodies (turquoise) for samples collected from previously infected and vaccinated individuals. (A) GenScrip sVNT, (B) Dynamiker sVNT, (C) Mindray NTAb sVNT. Presented data show the log-transformed values of quantitative results obtained by each assay.

Figure 4

Correlation and linear regression analysis between pVNT and GenScript sVNT (A), Dynamiker sVNT (B), and Mindray NTAb sVNT (C) using samples collected from SARS-CoV-2 infected patients and vaccinated individuals. Correlation and linear regression analyses were performed in GraphPad Prism using Pearson's correlation coefficients (r) and R². Statistical significance was calculated using the two-tailed test. Presented data are the log-transformed values of neutralization (%) by pVNT and nAb results obtained by each sVNT. The dashed lines indicate the 95% confidence intervals of the linear regression plots.

Figure 5

Empirical receiving operating characteristic (ROC) curve analysis for estimating optimal thresholds for sVNT assays targeting nAbs against SARS-CoV-2. (A) GenScript sVNT ROC curve in SARS-CoV-2 infected patients (n = 105). (B) GenScript sVNT ROC curve in COVID-19 vaccinated individuals (n = 24). (C) Dynamiker sVNT ROC curve in SARS-CoV-2 infected patients (n = 127). (D) Dynamiker sVNT ROC curve in COVID-19 vaccinated individuals (n = 156). (E) Mindray sVNT ROC curve in SARS-CoV-2 infected patients (n = 52). (F) Mindray sVNT ROC curve in COVID-19 vaccinated individuals (n = 155). The sensitivity and specificity values correspond to the plotted points which were used to estimate the area under the curve (AUC) and p-value for each curve.