Comparative performance of SARS-CoV-2 lateral flow antigen tests and association with detection of infectious virus in clinical specimens: a single-centre laboratory evaluation study

Abstract

Background: Lateral flow devices (LFDs) for rapid antigen testing are set to become a cornerstone of SARS-CoV-2 mass community testing, although their reduced sensitivity compared with PCR has raised questions of how well they identify infectious cases. Understanding their capabilities and limitations is, therefore, essential for successful implementation. We evaluated six commercial LFDs and assessed their correlation with infectious virus culture and PCR cycle threshold (Ct) values.

Methods: In a single-centre, laboratory evaluation study, we did a head-to-head comparison of six LFDs commercially available in the UK: Innova Rapid SARS-CoV-2 Antigen Test, Spring Healthcare SARS-CoV-2 Antigen Rapid Test Cassette, E25Bio Rapid Diagnostic Test, Encode SARS-CoV-2 Antigen Rapid Test Device, SureScreen COVID-19 Rapid Antigen Test Cassette, and SureScreen COVID-19 Rapid Fluorescence Antigen Test. We estimated the specificities and sensitivities of the LFDs using stored naso-oropharyngeal swabs collected at St Thomas' Hospital (London, UK) for routine diagnostic SARS-CoV-2 testing by real-time RT-PCR (RT-rtPCR). Swabs were from inpatients and outpatients from all departments of St Thomas' Hospital, and from health-care staff (all departments) and their household contacts. SARS-CoV-2-negative swabs from the same population (confirmed by RT-rtPCR) were used for comparative specificity determinations. All samples were collected between March 23 and Oct 27, 2020. We determined the limit of detection (LOD) for each test using viral plaque-forming units (PFUs) and viral RNA copy numbers of laboratory-grown SARS-CoV-2. Additionally, LFDs were selected to assess the correlation of antigen test result with RT-rtPCR Ct values and positive viral culture in Vero E6 cells. This analysis included longitudinal swabs from five infected inpatients with varying disease severities. Furthermore, the sensitivities of available LFDs were assessed in swabs (n=23; collected from Dec 4, 2020, to Jan 12, 2021) confirmed to be positive (RT-rtPCR and whole-genome sequencing) for the B.1.1.7 variant, which was the dominant genotype in the UK at the time of study completion.

Findings: All LFDs showed high specificity (≥98·0%), except for the E25Bio test (86·0% [95% CI 77·9-99·9]), and most tests reliably detected 50 PFU/test (equivalent SARS-CoV-2 N gene Ct value of 23·7, or RNA copy number of 3 × 10⁶/mL). Sensitivities of the LFDs on clinical samples ranged from 65·0% (55·2-73·6) to 89·0% (81·4-93·8). These sensitivities increased to greater than 90% for samples with Ct values of lower than 25 for all tests except the SureScreen fluorescence (SureScreen-F) test. Positive virus culture was identified in 57 (40·4%) of 141 samples; 54 (94·7%) of the positive cultures were from swabs with Ct values lower than 25. Among the three LFDs selected for detailed comparisons (the tests with highest sensitivity [Innova], highest specificity [Encode], and alternative technology [SureScreen-F]), sensitivity of the LFDs increased to at least 94·7% when only including samples with detected viral growth. Longitudinal studies of RT-rtPCR-positive samples (tested with Innova, Encode, and both SureScreen-F and the SureScreen visual [SureScreen-V] test) showed that most of the tests identified all infectious samples as positive. Test performance (assessed for Innova and SureScreen-V) was not affected when reassessed on swabs positive for the UK variant B.1.1.7.

Interpretation: In this comprehensive comparison of antigen LFDs and virus infectivity, we found a clear relationship between Ct values, quantitative culture of infectious virus, and antigen LFD positivity in clinical samples. Our data support regular testing of target groups with LFDs to supplement the current PCR testing capacity, which would help to rapidly identify infected individuals in situations in which they would otherwise go undetected.

Funding: King's Together Rapid COVID-19, Medical Research Council, Wellcome Trust, Huo Family Foundation, UK Department of Health, National Institute for Health Research Comprehensive Biomedical Research Centre.

Figure 1

Comparative sensitivity of six commercial SARS-CoV-2 rapid antigen tests (A) Heatmap comparison of lower limit of detection. (B) Association between PFUs/mL and Ct result from RT-rtPCR for the SARS-CoV-2 N gene (N Ct value). Error bars represent SD of three independent experiments. (C) Association between RNA copy number and N Ct values. Copy number per mL was derived from N RT-rtPCR on the Zeptometrix RNA standard (consisting of purified, inactivated viral particles of known RNA copy number per mL). In (B) and (C), points were fitted with a linear regression, with the equation of the line and the R² value shown. Horizontal dashed lines denote Ct values of 25 and 28, as the threshold cutoffs used for sensitivity determinations. (D) Tests were evaluated in head-to-head comparisons on a panel of 100 SARS-CoV-2-positive naso-oropharyngeal swabs. Bars denote the N Ct result for each swab, in ascending order, with the antigen test results for each sample directly below each bar presented as a heatmap. Sensitivity determinations from this sample set are shown on the right. Ct value cutoffs of 25 and 28, corresponding to 1·49 × 10⁶ and 1·65 × 10⁵ RNA copies per mL, or 400 and 50 PFUs/mL, respectively, are indicated, with corresponding sensitivity values for each test at each threshold on the right. PFU=plaque-forming unit. SureScreen-F=SureScreen fluorescent. SureScreen-V=SureScreen visual. N=SARS-CoV-2 nucleocapsid. Ct=cycle threshold. RT-rtPCR=real-time RT-PCR.

Figure 2

Comparison of Ct value, infectivity, and rapid antigen test result (A) Infectivity, N Ct value, and antigen test results for three commercial tests. Bars show the N Ct result for each swab in ascending order, coloured according to whether virus was cultured from the sample. Antigen test results for each sample are shown below each bar. Sensitivity determinations for the sample set are shown on the right. (B) Direct viral titres of swabs determined by plaque assay and compared with N Ct value. The linear relationship between Ct value and log₁₀ concentration of infectious virus in 57 samples is shown. (C) Ct results from (A) and RNA copies per mL plotted against days from symptom onset. Points are coloured according to virus growth (left graph; n=141) or the score derived from the Innova antigen test (right graph; n=110). (D) Longitudinal examples of infectivity, antigen test positivity, and Ct values for five infected individuals with varying COVID-19 severities., Bars show the N Ct value for each sample, shaded according to virus culture result. Antigen test results for each sample are shown below the bars. N=SARS-CoV-2 nucleocapsid. Ct=cycle threshold. SureScreen-F=SureScreen fluorescent. SureScreen-V=SureScreen visual. *Less than total sample due to limited sample volume or number of test kits available. †Time post-onset of symptoms for symptomatic individuals; time from the first positive PCR test for asymptomatic individuals. ‡Sufficient sample volume and test availability for inclusion in the longitudinal study for completeness of the dataset.

Figure 3

Comparative evaluation of antigen test sensitivity for the B.1.1.7 variant versus non-B.1.1.7 variant Combined naso-oropharyngeal swabs were obtained from 23 individuals with confirmed SARS-CoV-2 B.1.1.7 infection (December, 2020, and January, 2021) and compared with samples from before the variant was widely circulating in the UK population (April and September, 2020). All swabs were matched for N Ct values, shown in ascending order in the graph, and tested on Innova and SureScreen-V rapid antigen tests. N=SARS-CoV-2 nucleocapsid. Ct=cycle threshold. SureScreen-V=SureScreen visual.