Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Multicenter Study
. 2023 Sep;26(3):412-429.
doi: 10.1007/s10729-023-09647-2. Epub 2023 Jul 10.

Covid-19 triage in the emergency department 2.0: how analytics and AI transform a human-made algorithm for the prediction of clinical pathways

Christina C Bartenschlager  1   2   3 , Milena Grieger  1 , Johanna Erber  4 , Tobias Neidel  3 , Stefan Borgmann  5 , Jörg J Vehreschild  6   7   8 , Markus Steinbrecher  9 , Siegbert Rieg  10 , Melanie Stecher  7   8 , Christine Dhillon  11 , Maria M Ruethrich  12 , Carolin E M Jakob  7   8 , Martin Hower  13 , Axel R Heller  3 , Maria Vehreschild  14 , Christoph Wyen  15   16 , Helmut Messmann  9 , Christiane Piepel  17 , Jens O Brunner  18   19   20 , Frank Hanses  21 , Christoph Römmele  9   11 ; LEOSS study group
Collaborators, Affiliations
Multicenter Study

Covid-19 triage in the emergency department 2.0: how analytics and AI transform a human-made algorithm for the prediction of clinical pathways

Christina C Bartenschlager et al. Health Care Manag Sci. 2023 Sep.

Abstract

The Covid-19 pandemic has pushed many hospitals to their capacity limits. Therefore, a triage of patients has been discussed controversially primarily through an ethical perspective. The term triage contains many aspects such as urgency of treatment, severity of the disease and pre-existing conditions, access to critical care, or the classification of patients regarding subsequent clinical pathways starting from the emergency department. The determination of the pathways is important not only for patient care, but also for capacity planning in hospitals. We examine the performance of a human-made triage algorithm for clinical pathways which is considered a guideline for emergency departments in Germany based on a large multicenter dataset with over 4,000 European Covid-19 patients from the LEOSS registry. We find an accuracy of 28 percent and approximately 15 percent sensitivity for the ward class. The results serve as a benchmark for our extensions including an additional category of palliative care as a new label, analytics, AI, XAI, and interactive techniques. We find significant potential of analytics and AI in Covid-19 triage regarding accuracy, sensitivity, and other performance metrics whilst our interactive human-AI algorithm shows superior performance with approximately 73 percent accuracy and up to 76 percent sensitivity. The results are independent of the data preparation process regarding the imputation of missing values or grouping of comorbidities. In addition, we find that the consideration of an additional label palliative care does not improve the results.

Keywords: Artificial intelligence; Clinical decision making; Covid-19 triage; Machine learning; Predictive analytics.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no competing interests.

Figures

Fig. 1
Fig. 1
Base triage algorithm (TA) according to Pin et al. [39]
Fig. 2
Fig. 2
Extended triage algorithm (TAE). Yellow boxes highlight the differences compared to TA (see Fig. 1). TAE Scores for laboratory values, vital signs, demographic values, and comorbidities are shown in Table 4
Fig. 3
Fig. 3
Integrated triage algorithm (ITA). ITA Scores for ICU, ward, and outpatient are shown in Table 5
Fig. 4
Fig. 4
Comparison of the algorithms based on the accuracy (upper), ROC AUC (middle) and radar charts (lower) for data sets with 3 labels (left hand side) and four labels (right hand side). The respective boxplot represents the distribution of accuracy for the different data preparations. Both radar charts compare sensitivities, precision, and accuracies of the different algorithms. On the left-hand side, the XGB is used for all machine learning models, because of the similar performance
Fig. 5
Fig. 5
Comparison of the algorithms based on the recall of ICU. The respective boxplot represents the distribution of recalls for the different data preparations with 3 labels
See this image and copyright information in PMC

References

    1. Arballa N, Al-Turaiki I (2021) Machine learning approaches in COVID-19 diagnosis, mortality, and severity risk prediction: a review. Informatics in Medicine Unlocked. Online First - PMC - PubMed
    1. Ardabili SF, Mosavi A, Ghamisi P, Ferdinand F, Varkonyi-Koczy AR, Reuter U, Rabczuk T, Atkinson PM. COVID-19 outbreak prediction with machine learning. Algorithms. 2020;13(10):1–36. doi: 10.3390/a13100249. - DOI
    1. Arrieta AB, Díaz-Rodríguez N, Del Ser J, Bennetot A, Tabik S, Barbado A, Garcia S, Gil-Lopez S, Molina D, Benjamins R, Chatila R, Herrera F. Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Inf Fusion. 2020;58:82–115. doi: 10.1016/j.inffus.2019.12.012. - DOI
    1. Azad-Khaneghah P, Neubauer N, Cruz AM, Liu L. Mobile health app usability and quality rating scales: a systematic review. Disabil Rehabil Assist Technol. 2021;16(7):712–721. doi: 10.1080/17483107.2019.1701103. - DOI - PubMed
    1. Bartenschlager CC, Ebel SS, Kling S, Vehreschild J, Zabel LT, Spinner CD, Schuler A, Heller AR, Borgmann S, Hoffmann R, Rieg S, Messmann H, Hower M, Brunner JO, Hanses F, Römmele C (2022) COVIDAL: a machine learning classifier for digital COVID-19 diagnosis in German hospitals, Working paper, University of Augsburg

Publication types