COVID-19 Outpatient Screening: a Prediction Score for Adverse Events

Abstract

Background: We sought to develop an automatable score to predict hospitalization, critical illness, or death in patients at risk for COVID-19 presenting for urgent care during the Massachusetts outbreak.

Methods: Single-center study of adult outpatients seen in respiratory illness clinics (RICs) or the emergency department (ED), including development (n = 9381, March 7-May 2) and prospective (n = 2205, May 3-14) cohorts. Data was queried from Partners Enterprise Data Warehouse. Outcomes were hospitalization, critical illness or death within 7 days. We developed the COVID-19 Acuity Score (CoVA) using automatically extracted data from the electronic medical record and learning-to-rank ordinal logistic regression modeling. Calibration was assessed using predicted-to-observed event ratio (E/O). Discrimination was assessed by C-statistics (AUC).

Results: In the development cohort, 27.3%, 7.2%, and 1.1% of patients experienced hospitalization, critical illness, or death, respectively; and in the prospective cohort, 26.1%, 6.3%, and 0.5%. CoVA showed excellent performance in the development cohort (concurrent validation) for hospitalization (E/O: 1.00, AUC: 0.80); for critical illness (E/O: 1.00, AUC: 0.82); and for death (E/O: 1.00, AUC: 0.87). Performance in the prospective cohort (prospective validation) was similar for hospitalization (E/O: 1.01, AUC: 0.76); for critical illness (E/O 1.03, AUC: 0.79); and for death (E/O: 1.63, AUC=0.93). Among 30 predictors, the top five were age, diastolic blood pressure, blood oxygen saturation, COVID-19 testing status, and respiratory rate.

Conclusions: CoVA is a prospectively validated automatable score to assessing risk for adverse outcomes related to COVID-19 infection in the outpatient setting.

Figure 1.

(A) Distributions of adverse events (AE) within 7 days after initial outpatient evaluation in the RIC/ED, binned by acuity score. Colors represent hospitalization, ICU/MV, or death. (B) Cumulative incidence of critical illness or death up to 17 days following initial evaluation, based on initial acuity score. Curves are computed based on cross validation in the development cohort (C,D,E,F) Calibration curves: predicted probability of adverse events vs. observed rate of adverse events. C (development cohort) and D (prospective validation cohort) show calibration for predicting hospitalization; E (development cohort) and F (prospective validation cohort) show calibration for predictions of critical illness or death. Overall calibration (E/O) and calibration slopes (CS) are reported in Table 2.