Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jul 12;36(27):e175.
doi: 10.3346/jkms.2021.36.e175.

Automatic Classification of the Korean Triage Acuity Scale in Simulated Emergency Rooms Using Speech Recognition and Natural Language Processing: a Proof of Concept Study

Dongkyun Kim #  1 , Jaehoon Oh #  2 , Heeju Im  3 , Myeongseong Yoon  2 , Jiwoo Park  2 , Joohyun Lee  4
Affiliations

Automatic Classification of the Korean Triage Acuity Scale in Simulated Emergency Rooms Using Speech Recognition and Natural Language Processing: a Proof of Concept Study

Dongkyun Kim et al. J Korean Med Sci. .

Abstract

Background: Rapid triage reduces the patients' stay time at an emergency department (ED). The Korean Triage Acuity Scale (KTAS) is mandatorily applied at EDs in South Korea. For rapid triage, we studied machine learning-based triage systems composed of a speech recognition model and natural language processing-based classification.

Methods: We simulated 762 triage cases that consisted of 18 classes with six types of the main symptom (chest pain, dyspnea, fever, stroke, abdominal pain, and headache) and three levels of KTAS. In addition, we recorded conversations between emergency patients and clinicians during the simulation. We used speech recognition models to transcribe the conversation. Bidirectional Encoder Representation from Transformers (BERT), support vector machine (SVM), random forest (RF), and k-nearest neighbors (KNN) were used for KTAS and symptom classification. Additionally, we evaluated the Shapley Additive exPlanations (SHAP) values of features to interpret the classifiers.

Results: The character error rate of the speech recognition model was reduced to 25.21% through transfer learning. With auto-transcribed scripts, support vector machine (area under the receiver operating characteristic curve [AUROC], 0.86; 95% confidence interval [CI], 0.81-0.9), KNN (AUROC, 0.89; 95% CI, 0.85-0.93), RF (AUROC, 0.86; 95% CI, 0.82-0.9) and BERT (AUROC, 0.82; 95% CI, 0.75-0.87) achieved excellent classification performance. Based on SHAP, we found "stress", "pain score point", "fever", "breath", "head" and "chest" were the important vocabularies for determining KTAS and symptoms.

Conclusion: We demonstrated the potential of an automatic KTAS classification system using speech recognition models, machine learning and BERT-based classifiers.

Keywords: Classification; Deep Learning; Machine Learning; Natural Language Processing; Triage.

PubMed Disclaimer

Conflict of interest statement

The authors have no potential conflicts of interest to disclose.

Figures

Fig. 1
Fig. 1. Overall flow of the classification system in this study.
KTAS = Korean Triage and Acuity Scale, CER = character error rate, BERT = Bidirectional Encoder Representations from Transformers, SVM = support vector machine, KNN = k-nearest neighbors, RF = random forest, AUROC = area under the receiver operating characteristic curve, SHAP = Shapley Additive exPlanations.
See this image and copyright information in PMC

References

    1. National Emergency Medical Center (Korea) 2019 Annual report of Korean emergency medicine. [Updated 2020]. [Accessed January 13, 2021]. https://www.e-gen.or.kr/nemc/statistics_annual_report.do.
    1. Bernstein SL, Aronsky D, Duseja R, Epstein S, Handel D, Hwang U, et al. The effect of emergency department crowding on clinically oriented outcomes. Acad Emerg Med. 2009;16(1):1–10. - PubMed
    1. Stang AS, Crotts J, Johnson DW, Hartling L, Guttmann A. Crowding measures associated with the quality of emergency department care: a systematic review. Acad Emerg Med. 2015;22(6):643–656. - PubMed
    1. Park J, Lim T. Korean Triage and Acuity Scale (KTAS) J Korean Soc Emerg Med. 2017;28(6):547–551.
    1. Ryu JH, Min MK, Lee DS, Yeom SR, Lee SH, Wang IJ, et al. Changes in relative importance of the 5-level triage system, Korean Triage and Acuity Scale, for the disposition of emergency patients induced by forced reduction in its level number: a multi-center registry based retrospective cohort study. J Korean Med Sci. 2019;34(14):e114. - PMC - PubMed