Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Oct;102(10):001653.
doi: 10.1099/jgv.0.001653.

Head-to-head evaluation of seven different seroassays including direct viral neutralisation in a representative cohort for SARS-CoV-2

Affiliations

Head-to-head evaluation of seven different seroassays including direct viral neutralisation in a representative cohort for SARS-CoV-2

Laura Olbrich et al. J Gen Virol. 2021 Oct.

Abstract

A number of seroassays are available for SARS-CoV-2 testing; yet, head-to-head evaluations of different testing principles are limited, especially using raw values rather than categorical data. In addition, identifying correlates of protection is of utmost importance, and comparisons of available testing systems with functional assays, such as direct viral neutralisation, are needed.We analysed 6658 samples consisting of true-positives (n=193), true-negatives (n=1091), and specimens of unknown status (n=5374). For primary testing, we used Euroimmun-Anti-SARS-CoV-2-ELISA-IgA/IgG and Roche-Elecsys-Anti-SARS-CoV-2. Subsequently virus-neutralisation, GeneScriptcPass, VIRAMED-SARS-CoV-2-ViraChip, and Mikrogen-recomLine-SARS-CoV-2-IgG were applied for confirmatory testing. Statistical modelling generated optimised assay cut-off thresholds. Sensitivity of Euroimmun-anti-S1-IgA was 64.8%, specificity 93.3% (manufacturer's cut-off); for Euroimmun-anti-S1-IgG, sensitivity was 77.2/79.8% (manufacturer's/optimised cut-offs), specificity 98.0/97.8%; Roche-anti-N sensitivity was 85.5/88.6%, specificity 99.8/99.7%. In true-positives, mean and median Euroimmun-anti-S1-IgA and -IgG titres decreased 30/90 days after RT-PCR-positivity, Roche-anti-N titres decreased significantly later. Virus-neutralisation was 80.6% sensitive, 100.0% specific (≥1:5 dilution). Neutralisation surrogate tests (GeneScriptcPass, Mikrogen-recomLine-RBD) were >94.9% sensitive and >98.1% specific. Optimised cut-offs improved test performances of several tests. Confirmatory testing with virus-neutralisation might be complemented with GeneScriptcPassTM or recomLine-RBD for certain applications. Head-to-head comparisons given here aim to contribute to the refinement of testing strategies for individual and public health use.

Keywords: COVID-19; RBD; SARS-CoV-2; antibody; nucleocapsid; serology; spike; virus neutralisation.

PubMed Disclaimer

Conflict of interest statement

AW and MH report personal fees and non-financial support from Roche Diagnostics, LO reports non-financial support from Roche Diagnostics. AW, MH and LO report non-financial support from Euroimmun, non-financial support from Viramed, non-financial support from Mikrogen. AW, MH, LO report grants, non-financial support and other from German Centre for Infection Research DZIF, grants and non-financial support from Government of Bavaria, non-financial support from BMW, non-financial support from Munich Police, non-financial support and other from Accenture. JH reports grants from German Federal Ministry of Education and Research, during the conduct of the study. MH and AW report personal fees and non-financial support from Dr. Box-Betrobox, non-financial support from Dr. Becker MVZ during the conduct of the study. In addition, MH, AW, MB have a patent on a sample system for sputum diagnostics of SARS-CoV-2 pending. AW is involved in other different patents and companies not in relation with the serology of SARS-CoV-2. AW reports personal fees and other from Haeraeus Sensors, non-financial support from Bruker Daltonics, all of which are outside the submitted work, and non-related to SARS-CoV-2. MB is an authorised representative partner of Dr. Becker MVZ.

Figures

Fig. 1.
Fig. 1.
Performance of primary tests. Results of primary tests for true-negatives (blue), true-positives (orange), and individuals with unknown SARS-CoV-2 status (grey). Absolute number of subjects (count/y-axis) and distribution of raw values (x-axis) measured for EI-S1-IgA (left), EI-S1-IgG (centre), and Ro-N-Ig (right). Dotted lines mark the manufacturer’s cut-off value (between indeterminate and positive for EI, and between negative and positive in Ro). Dashed lines mark the optimised cut-off value as determined in this study (overlapping with the dotted line for EI-S1-IgA). Orange and blue solid lines represent the percentage of test results for true-positives and true-negatives above (blue) or below (orange) the value on the x-axes, respectively. Orange and blue numbers give the percentages of true-positives and true-negatives that were correctly detected by the test (within brackets: manufacturers' cut-offs; without brackets: optimised cut-offs).
Fig. 2.
Fig. 2.
Comparison of primary tests. Results of primary tests compared to ground truth for true-negatives (blue), true-positives (orange), and individuals with unknown SARS-CoV-2 status (grey). The dotted lines represent the manufacturer’s cut-offs, the dashed lines the optimised cut-offs defined within this study. (a) Pairwise scatter plots for EI-S1-IgA vs. EI-S1-IgG (left; n=6657), and Ro-N-Ig vs. EI-S1-IgG (right; n=6636). Percentages in orange indicate fractions of true-positives in the respective quadrant with respect to all true-positives; blue for true-negatives. Percentages were calculated using the optimised cut-off. (b) Parallel coordinate plot of the same three tests.
Fig. 3.
Fig. 3.
Time dependence in primary tests for RT-PCR true-positives. Titre values of 187 true-positives with available data on time between RT-PCR and blood sampling for (a) EI-S1-IgA, (b) EI-S1-IgG, and (c) Ro-N-Ig. The read-outs were categorised according to the time after positive RT-PCR (<30 days, 30–90 days and >90 days). Plots show the individual read-out (orange dots), a density estimate (orange area), the 25-,50- and 75-percentiles (black boxes), and the means (black dots). Counts n (n refer to the number of observations above/below manufacturer’s (optimised) cut-off for each of the temporal groups). Pairwise differences are considered only after adjusting for multiple testing and can be found in Table S4. Mean values (mv) and median values (med) are given for each group.
Fig. 4.
Fig. 4.
Confirmatory tests. Results of confirmatory tests compared to ground truth for true-negatives (blue), true-positives (orange), and individuals with unknown SARS-CoV-2 status (grey). Black dotted and dashed lines represent the manufacturers and the optimised cut-offs, respectively. Orange/blue numbers indicate percentages of true-positives/-negatives correctly detected by the test using the respective cut-offs (identical in a, b, d). Distribution of results of NT (a) and GS-cPass (b). Distribution of IgG results of the VC-array (c) and the MG-line blot (d). Bar charts below violin plots represent information on the categorical part of the values below linear range. Grey numbers give the percentages of positive samples with unknown SARS-CoV-2 as determined by the manufacturers and optimised cut-offs. Percentages were calculated over the total number of samples of unknown SARS-CoV-2 status with available test results.
Fig. 5.
Fig. 5.
Comparison of confirmatory tests. Comparison of confirmatory tests for true-negatives (blue), true-positives (orange), and individuals with unknown SARS-CoV-2 status (grey). At the top, in black, total number of cases (n) for each NT category. (a) Association between the categorical endpoint of NT and the continuous results of GS-cPass (n=354). (b) Association between the categorical endpoint of NT and the continuous results of MG-RBD (n=272). (c) Association between GS-cPass and MG-RBD (n=272). The solid black line represents a linear regression for the positive measurements.
Fig. 6.
Fig. 6.
Comparison of primary tests (EI-S1-IgG, Ro-N-Ig) with confirmatory tests (NT, GS-cPass MG-RBD, MG-N). Comparison of EI-S1-IgG and Ro-N-Ig with confirmatory tests for true-negatives (blue), true-positives (orange), and individuals with unknown SARS-CoV-2 status (grey) using the optimised cut-offs. The solid black line represents a linear regression for the positive measurements. (a) From left to right, association of EI-S1-IgG with the confirmatory test NT (n=354), GS-cPass (n=361), MG-RBD (n=272) and MG-N (n=355). We observed a population in the upper left quadrant, clearly negative in the confirmatory tests GS-cPass, MG-RBD and MG-N. (b) From left to right, association of Ro-N-Ig with the confirmatory test NT (n=362), GS-cPass (n=273), MG-RBD (n=354), and MG-N n=354).

References

    1. Huang C, Wang Y, Li X, Ren L, Zhao J, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. The Lancet. 2020;395:497–506. doi: 10.1016/S0140-6736(20)30183-5. - DOI - PMC - PubMed
    1. Amanat F, Stadlbauer D, Strohmeier S, Nguyen THO, Chromikova V, et al. A serological assay to detect SARS-CoV-2 seroconversion in humans. Nat Med. 2020;26:1033–1036. doi: 10.1038/s41591-020-0913-5. - DOI - PMC - PubMed
    1. Chen X, Pan Z, Yue S, Yu F, Zhang J, et al. Disease severity dictates SARS-CoV-2-specific neutralising antibody responses in COVID-19. Signal Transduct Target Ther. 2020;5:180. doi: 10.1038/s41392-020-00301-9. - DOI - PMC - PubMed
    1. Meyer B, Torriani G, Yerly S, Mazza L, Calame A, et al. Validation of a commercially available SARS-CoV-2 serological immunoassay. Clin Microbiol Infect. 2020;26:1386–1394. doi: 10.1016/j.cmi.2020.06.024. - DOI - PMC - PubMed
    1. (FIND) FfIND SARS-CoV-2 diagnostic pipeline. 2020. https://www.finddx.org/covid-19/pipeline

Publication types