Derivation With Internal Validation of a Multivariable Predictive Model to Predict COVID-19 Test Results in Emergency Department Patients

Abstract

Objectives: The COVID-19 pandemic has placed acute care providers in demanding situations in predicting disease given the clinical variability, desire to cohort patients, and high variance in testing availability. An approach to stratifying patients by likelihood of disease based on rapidly available emergency department (ED) clinical data would offer significant operational and clinical value. The purpose of this study was to develop and internally validate a predictive model to aid in the discrimination of patients undergoing investigation for COVID-19.

Methods: All patients greater than 18 years presenting to a single academic ED who were tested for COVID-19 during this index ED evaluation were included. Outcome was defined as the result of COVID-19 polymerase chain reaction (PCR) testing during the index visit or any positive result within the following 7 days. Variables included chest radiograph interpretation, disease-specific screening questions, and laboratory data. Three models were developed with a split-sample approach to predict outcome of the PCR test utilizing logistic regression, random forest, and gradient-boosted decision tree methods. Model discrimination was evaluated comparing area under the receiver operator curve (AUC) and point statistics at a predefined threshold.

Results: A total of 1,026 patients were included in the study collected between March and April 2020. Overall, there was disease prevalence of 9.6% in the population under study during this time frame. The logistic regression model was found to have an AUC of 0.89 (95% confidence interval [CI] = 0.84 to 0.94) when including four features: exposure history, temperature, white blood cell count (WBC), and chest radiograph result. Random forest method resulted in AUC of 0.86 (95% CI = 0.79 to 0.92) and gradient boosting had an AUC of 0.85 (95% CI = 0.79 to 0.91). With a consistently held negative predictive value, the logistic regression model had a positive predictive value of 0.29 (0.2-0.39) compared to 0.2 (0.14-0.28) for random forest and 0.22 (0.15-0.3) for the gradient-boosted method.

Conclusion: The derived predictive models offer good discriminating capacity for COVID-19 disease and provide interpretable and usable methods for those providers caring for these patients at the important crossroads of the community and the health system. We found utilization of the logistic regression model utilizing exposure history, temperature, WBC, and chest X-ray result had the greatest discriminatory capacity with the most interpretable model. Integrating a predictive model-based approach to COVID-19 testing decisions and patient care pathways and locations could add efficiency and accuracy to decrease uncertainty.

Figure 1

Visualization of missing data in training and validation data. AST/ALT = alanine aminotransferase/aspartate aminotransferase; MAP = mean arterial pressure; WBC =white blood cell count.

Figure 2

Area under the receiver operator curve of the three derived models.

Figure 3

Box plots for the three derived models with predicted probabilities stratified by the true outcome.