Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Randomized Controlled Trial
. 2024 May 30:12:1391906.
doi: 10.3389/fpubh.2024.1391906. eCollection 2024.

Assessing the utility of artificial intelligence throughout the triage outpatients: a prospective randomized controlled clinical study

Xiaoni Liu #  1   2 , Rui Lai #  3 , Chaoling Wu #  4 , Changjian Yan  5 , Zhe Gan  6 , Yaru Yang  1 , Xiangtai Zeng  7 , Jin Liu  8 , Liangliang Liao  8 , Yuansheng Lin  9 , Hongmei Jing  1 , Weilong Zhang  1
Affiliations
Randomized Controlled Trial

Assessing the utility of artificial intelligence throughout the triage outpatients: a prospective randomized controlled clinical study

Xiaoni Liu et al. Front Public Health. .

Abstract

Currently, there are still many patients who require outpatient triage assistance. ChatGPT, a natural language processing tool powered by artificial intelligence technology, is increasingly utilized in medicine. To facilitate and expedite patients' navigation to the appropriate department, we conducted an outpatient triage evaluation of ChatGPT. For this evaluation, we posed 30 highly representative and common outpatient questions to ChatGPT and scored its responses using a panel of five experienced doctors. The consistency of manual triage and ChatGPT triage was assessed by five experienced doctors, and statistical analysis was performed using the Chi-square test. The expert ratings of ChatGPT's answers to these 30 frequently asked questions revealed 17 responses earning very high scores (10 and 9.5 points), 7 earning high scores (9 points), and 6 receiving low scores (8 and 7 points). Additionally, we conducted a prospective cohort study in which 45 patients completed forms detailing gender, age, and symptoms. Triage was then performed by outpatient triage staff and ChatGPT. Among the 45 patients, we found a high level of agreement between manual triage and ChatGPT triage (consistency: 93.3-100%, p<0.0001). We were pleasantly surprised to observe that ChatGPT's responses were highly professional, comprehensive, and humanized. This innovation can help patients win more treatment time, improve patient diagnosis and cure rates, and alleviate the pressure of medical staff shortage.

Keywords: AI; ChatGPT; artificial intelligence; triage; triage outpatients.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
The reviewer’s evaluation of ChatGPT outpatient triage. (A) The reviewer’s rating of ChatGPT’s answers. (B) Reviewers assessed the consistency of manual and ChatGPT triage. Q, Questions. D, Doctor. Yes: ChatGPT triage in the same number as manual triage. No: Number of ChatGPT triage was inconsistent with manual triage. ****p<0.0001. The chi-square test was used in this study.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Yao C. Statistical bulletin of China's health development in 2022. Health News. (2023) 4:001.
    1. Christ M, Bingisser R, Nickel CH. Emergency triage. An overview. Dtsch Med Wochenschr. (2016) 141:329–35. doi: 10.1055/s-0041-109126, PMID: - DOI - PubMed
    1. Dondi A, Calamelli E, Scarpini S. Triage grading and correct diagnosis are critical for the emergency treatment of anaphylaxis Children (Basel) (2022) 9:12. doi: 10.3390/children9121794 - DOI - PMC - PubMed
    1. Christian MD. Triage. Crit Care Clin. (2019) 35:575–89. doi: 10.1016/j.ccc.2019.06.009, PMID: - DOI - PMC - PubMed
    1. Xie W, Cao X, Dong H, Liu Y. The use of smartphone-based triage to reduce the rate of outpatient error registration: cross-sectional study. JMIR Mhealth Uhealth. (2019) 7:e15313. doi: 10.2196/15313, PMID: - DOI - PMC - PubMed

Publication types