Multicenter Technical Validation of 30 Rapid Antigen Tests for the Detection of SARS-CoV-2 (VALIDATE)

Abstract

During COVID19 pandemic, SARS-CoV-2 rapid antigen tests (RATs) were marketed with minimal or no performance data. We aimed at closing this gap by determining technical sensitivities and specificities of 30 RATs prior to market release. We developed a standardized technical validation protocol and assessed 30 RATs across four diagnostic laboratories. RATs were tested in parallel using the Standard Q^® (SD Biosensor/Roche) assay as internal reference. We used left-over universal transport/optimum media from nasopharyngeal swabs of 200 SARS-CoV-2 PCR-negative and 100 PCR-positive tested patients. Transport media was mixed with assay buffer and applied to RATs according to manufacturer instructions. Sensitivities were determined according to viral loads. Specificity of at least 99% and sensitivity of 95%, 90%, and 80% had to be reached for 10⁷, 10⁶, 10⁵ virus copies/mL, respectively. Sensitivities ranged from 43.5% to 98.6%, 62.3% to 100%, and 66.7% to 100% at 10⁵, 10⁶, 10⁷ copies/mL, respectively. Automated assay readers such as ExDia or LumiraDx showed higher performances. Specificities ranged from 88.8% to 100%. Only 15 of 30 (50%) RATs passed our technical validation. Due to the high failure rate of 50%, mainly caused by lack of sensitivity, we recommend a thorough validation of RATs prior to market release.

Keywords: COVID-19; SARS-CoV-2; diagnostics; rapid antigen test; virus testing.

Figure 1

Box-plots of Ct values distributions according to different batches of validation series. Figure 1 legend. RATs were grouped according to the validation series. For each series, there was one internal reference (Standard Q^®, white box-plot). Black solid lines represent the Ct medians, box-plots and whiskers represent the Ct values distribution. RATs results are displayed on the x-axis: Negative or Positive. Cycle thresholds (Ct) are shown on the y-axis.

Figure 2

RATs sensitivity rates for viral loads above 105 copies/mL compared to the internal reference test. Figure 2 legend. Green and red dots represent the assays passing and not passing FOPH validation criteria, respectively; orange dots represent the assays validated using the non-inferiority criteria to the internal reference. The solid line represents the internal reference (Standard Q^®, SD Biosensor/Roche). Dashed horizontal lines represent the limits of difference in percentage within which sensitivity rates’ variations were considered acceptable compared to the IR. The vertical dashed line coincides with 80% cut-off, which was considered the minimal sensitivity threshold for FOPH validation above 105 copies/mL of viral load. ^: this test did not pass the validation criteria for insufficient sensitivity at viral loads above 106–107 copies/mL. °: this test did not pass the validation because of lack of specificity.

Figure 3

Comparative cumulated sensitivity curves for all rapid antigen tests (RATs) evaluated in the study. Figure 3 legend. (A). All RATs passing the validation criteria. (B). All RATs failing the validation. (C). RATs with an automated reader (Exdia from Precision Biosensor and LumiraDx assay from LumiraDx, Alloa) showed higher sensitivity performances compared to the internal reference (Roche). (D). Sensitivity curves comparing all the internal reference tests (Roche) used along each series showing the inter-laboratory and inter-series variability. RAT: rapid antigen test. On the x-axis there is the number of Ct; horizontal lines represent the Ct cut-offs considered for the validation criteria. Horizontal dotted lines and the three arrows represent the threshold of sensitivity rates to be considered for validation at different Ct cut-offs.