Comparison of four commercial, automated antigen tests to detect SARS-CoV-2 variants of concern

Abstract

A versatile portfolio of diagnostic tests is essential for the containment of the severe acute respiratory syndrome coronavirus type 2 (SARS-CoV-2) pandemic. Besides nucleic acid-based test systems and point-of-care (POCT) antigen (Ag) tests, quantitative, laboratory-based nucleocapsid Ag tests for SARS-CoV-2 have recently been launched. Here, we evaluated four commercial Ag tests on automated platforms and one POCT to detect SARS-CoV-2. We evaluated PCR-positive (n = 107) and PCR-negative (n = 303) respiratory swabs from asymptomatic and symptomatic patients at the end of the second pandemic wave in Germany (February-March 2021) as well as clinical isolates EU1 (B.1.117), variant of concern (VOC) Alpha (B.1.1.7) or Beta (B.1.351), which had been expanded in a biosafety level 3 laboratory. The specificities of automated SARS-CoV-2 Ag tests ranged between 97.0 and 99.7% (Lumipulse G SARS-CoV-2 Ag (Fujirebio): 97.03%, Elecsys SARS-CoV-2 Ag (Roche Diagnostics): 97.69%; LIAISON^® SARS-CoV-2 Ag (Diasorin) and SARS-CoV-2 Ag ELISA (Euroimmun): 99.67%). In this study cohort of hospitalized patients, the clinical sensitivities of tests were low, ranging from 17.76 to 52.34%, and analytical sensitivities ranged from 420,000 to 25,000,000 Geq/ml. In comparison, the detection limit of the Roche Rapid Ag Test (RAT) was 9,300,000 Geq/ml, detecting 23.58% of respiratory samples. Receiver-operating-characteristics (ROCs) and Youden's index analyses were performed to further characterize the assays' overall performance and determine optimal assay cutoffs for sensitivity and specificity. VOCs carrying up to four amino acid mutations in nucleocapsid were detected by all five assays with characteristics comparable to non-VOCs. In summary, automated, quantitative SARS-CoV-2 Ag tests show variable performance and are not necessarily superior to a standard POCT. The efficacy of any alternative testing strategies to complement nucleic acid-based assays must be carefully evaluated by independent laboratories prior to widespread implementation.

Keywords: Automated SARS-CoV-2 antigen test; Diagnostic test; Nucleocapsid protein; Sensitivity; Specificity; VOC.

Fig. 1

SARS-CoV-2 viral load distribution of respiratory samples included in the study. a Shown is the log10 viral load (Geq/ml) of all 107 SARS-CoV-2-positive patient samples, sorted by ascending magnitude from left to right. Each dot indicates one patient and the sample’s ID is indicated. b Depicted is the histogram of the viral load distribution by categorization of samples into defined log10 viral load value ranges. Each bar indicates the number of samples in the respective viral load range

Fig. 2

Analytical sensitivity of PCR-positive SARS-CoV-2 patient samples for quantitative SARS-CoV-2 Ag tests. a CLEIA from Fujirebio, b CLIA from Diasorin, c ELISA from Euroimmun and d ECLIA from Roche Diagnostics. The log₁₀ of quantified samples were plotted against the log₁₀ of the calculated viral loads. The horizontal dotted red line indicates the cutoff suggested by the manufacturer

Fig. 3

ROC analyses for quantitative SARS-CoV-2 Ag tests with a CLEIA, b CLIA, c ELISA and d ECLIA. The respective AUCs are depicted

Fig. 4

Limit of detection analyses of PCR-positive SARS-CoV-2 patient samples for quantitative SARS-CoV-2 Ag tests: a CLEIA, b CLIA, c ELISA, d ECLIA, and e the POCT RAT. The log10 viral load of quantified samples on the x-axis was plotted against a positive (+ 1) or negative (0) test outcome on the y-axis. For readability of the figure, slight normal jitter was added to the y-values. Red curves show logistic regressions of the viral load on the test outcome; horizontal dashed line shows 50% detection probability, whereas vertical dashed lines indicate log viral loads at which 50% (LoD₅₀) and 95% (LoD₉₅), respectively, of the samples are expected positive based on the regression results

Fig. 5

Quantification of VOCs using the quantitative SARS-CoV-2 Ag tests: a CLEIA, b CLIA, c ELISA and d ECLIA. The quantified samples were plotted against the calculated viral loads