Specificity and positive predictive value of SARS-CoV-2 nucleic acid amplification testing in a low-prevalence setting

Abstract

Objectives: When the prevalence of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) is low, many positive test results are false positives. Confirmatory testing reduces overdiagnosis and nosocomial infection and enables real-world estimates of test specificity and positive predictive value. This study estimates these parameters to evaluate the impact of confirmatory testing and to improve clinical diagnosis, epidemiological estimation and interpretation of vaccine trials.

Methods: Over 1 month we took all respiratory samples from our laboratory with a patient's first detection of SARS-CoV-2 RNA (Hologic Aptima SARS-CoV-2 assay or in-house RT-PCR platform), and repeated testing using two platforms. Samples were categorized by source, and by whether clinical details suggested COVID-19 or corroborative testing from another laboratory. We estimated specificity and positive predictive value using approaches based on maximum likelihood.

Results: Of 19 597 samples, SARS-CoV-2 RNA was detected in 107; 52 corresponded to first-time detection (0.27% of tests on samples without previous detection). Further testing detected SARS-CoV-2 RNA once or more ('confirmed') in 29 samples (56%), and failed to detect SARS-CoV-2 RNA ('not confirmed') in 23 (44%). Depending upon assumed parameters, point estimates for specificity and positive predictive value were 99.91-99.98% and 61.8-89.8% respectively using the Hologic Aptima SARS-CoV-2 assay, and 97.4-99.1% and 20.1-73.8% respectively using an in-house assay.

Conclusions: Nucleic acid amplification testing for SARS-CoV-2 is highly specific. Nevertheless, when prevalence is low a significant proportion of initially positive results fail to confirm, and confirmatory testing substantially reduces the detection of false positives. Omitting additional testing in samples with higher prior detection probabilities focuses testing where it is clinically impactful and minimizes delay.

Keywords: COVID-19 diagnostic testing; COVID-19 pandemic; Nucleic acid amplification techniques; SARS-CoV-2; Sensitivity and specificity.

Graphical abstract

Fig. 1

Platforms on which samples were tested, and outcomes of initial and confirmatory testing. Numbers in parentheses correspond to the numbers of samples at each stage of analysis. Bold endpoints correspond to the outcomes analysed to derive the main specificity, prevalence and positive predictive value results. Italic endpoints correspond to the outcomes analysed to derive results relating to samples from patients with previous SARS-CoV-2 RNA detection.

Fig. 2

Calculated specificity, prevalence of samples containing virus, and positive predictive value of the Hologic Aptima SARS-CoV-2 assay and the in-house assay. Calculated values are solid lines, with 95% confidence intervals (shaded regions enclosed by dashed lines), for assumed test sensitivities from 50% to 100%, and for the two extreme assumptions of all indeterminate samples being confirmed as positive, and all indeterminate samples not confirming as positive. A version of the Hologic Aptima SARS-CoV-2 plots with zoomed vertical scales may be found in web-only Supplementary Figure S1.

Fig. 3

Relative light unit output of the Hologic Panther analysis of the Aptima SARS-CoV-2 assay of samples initially determined to be first positives. Grouped by the reporting categories detailed in the flowchart in Fig. 1.

Fig. 4

Threshold cycles of the samples initially determined by the in-house assay to be first positives. Grouped by the reporting categories detailed in the flowchart in Fig. 1. The solid data points correspond to samples assayed when the in-house assay had a single target; the hollow points correspond to samples assayed when the in-house assay had two targets. In both cases only the RdRp target, and not the S target, was detected.