Sensitivity and specificity analyses of COVID-19 screening protocol for emergency medical services: A STARD-compliant population-based retrospective study

Abstract

During the novel coronavirus disease (COVID-19) pandemic, emergency medical services (EMS) has borne a huge burden in transporting emergency patients. However, the protocol's effect on identifying emergency patients who are likely to have COVID-19 is unknown. We aimed to evaluate the diagnostic accuracy of a prehospital COVID-19 screening protocol for EMS. We conducted this population-based retrospective study in Nara Prefecture, Japan. The Nara Prefectural Government implemented a screening protocol for COVID-19 comprising the following symptom criteria (fever, cough, sore throat, headache, malaise, dysgeusia, or anosmia) and epidemiological criteria (contact history with confirmed COVID-19 cases or people with upper respiratory symptoms, or travel to areas with high infection rate). A patient meeting at least one criterion of each class was considered positive. We evaluated all 51,351 patients from the regional EMS database of the Nara Prefecture (emergency Medical Alliance for Total Coordination of Healthcare) who were registered from June 15, 2020 to May 31, 2021 and had results of COVID-19 reverse transcription polymerase chain reaction (RT-PCR) tests. We assessed the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of this protocol. We also assessed how these outcomes changed by adding vital signs and conducted a 10-fold and 100-fold prevalence simulation. The screening protocol was used for 246/51351 patients (0.5%). Among them, 31 tested positive after EMS transportation. This protocol's sensitivity, specificity, PPV, and NPV were 40.8%, 99.6%, 12.6%, and 99.9%, respectively. With the addition of ≥2 vital signs (body temperature ≥37.5 °C, respiratory rate ≥20 breaths/minute, and oxygen saturation <90%), sensitivity and PPV changed to 61.8% and 1.0%, respectively, while NPV remained 99.9%. With a 10-fold and 100-fold increase in disease, the protocol PPV would be 59.0% and 94.3%, and NPV would be 99.1% and 90.7%, respectively, and with additional vital signs, PPV would be 8.9% and 53.1%, and NPV would be 99.4% and 93.2%, respectively. This COVID-19 screening protocol helped enable EMS transport for patients with COVID-19 with a PPV of 12.6%. Adding other vital sign variables may improve its diagnostic value if the prevalence rate increases.

Figure 1.

Study population.

Figure 2.

COVID-19 during the study period in Nara Prefecture, Japan. Patients that met the screening criteria (solid bars) and patients diagnosed with COVID-19 after emergency medical services (EMS) transportation (blank bars). The line illustrates the trend of COVID-19 cases in Nara Prefecture during the study period. COVID-19, coronavirus disease.