Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Apr 24;61(7):1158-1166.
doi: 10.1515/cclm-2023-0355. Print 2023 Jun 27.

Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI)

Janne Cadamuro  1 , Federico Cabitza  2   3 , Zeljko Debeljak  4   5 , Sander De Bruyne  6 , Glynis Frans  7 , Salomon Martin Perez  8 , Habib Ozdemir  9 , Alexander Tolios  10 , Anna Carobene  11 , Andrea Padoan  12
Affiliations
Free article

Potentials and pitfalls of ChatGPT and natural-language artificial intelligence models for the understanding of laboratory medicine test results. An assessment by the European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) Working Group on Artificial Intelligence (WG-AI)

Janne Cadamuro et al. Clin Chem Lab Med. .
Free article

Abstract

Objectives: ChatGPT, a tool based on natural language processing (NLP), is on everyone's mind, and several potential applications in healthcare have been already proposed. However, since the ability of this tool to interpret laboratory test results has not yet been tested, the EFLM Working group on Artificial Intelligence (WG-AI) has set itself the task of closing this gap with a systematic approach.

Methods: WG-AI members generated 10 simulated laboratory reports of common parameters, which were then passed to ChatGPT for interpretation, according to reference intervals (RI) and units, using an optimized prompt. The results were subsequently evaluated independently by all WG-AI members with respect to relevance, correctness, helpfulness and safety.

Results: ChatGPT recognized all laboratory tests, it could detect if they deviated from the RI and gave a test-by-test as well as an overall interpretation. The interpretations were rather superficial, not always correct, and, only in some cases, judged coherently. The magnitude of the deviation from the RI seldom plays a role in the interpretation of laboratory tests, and artificial intelligence (AI) did not make any meaningful suggestion regarding follow-up diagnostics or further procedures in general.

Conclusions: ChatGPT in its current form, being not specifically trained on medical data or laboratory data in particular, may only be considered a tool capable of interpreting a laboratory report on a test-by-test basis at best, but not on the interpretation of an overall diagnostic picture. Future generations of similar AIs with medical ground truth training data might surely revolutionize current processes in healthcare, despite this implementation is not ready yet.

Keywords: ChatGPT; artificial intelligence; chatbot; laboratory tests; natural language processing.

PubMed Disclaimer

Comment in

References

    1. Plebani, M, Laposata, M, Lippi, G. Driving the route of laboratory medicine: a manifesto for the future. Intern Emerg Med 2019;14:337–40. https://doi.org/10.1007/s11739-019-02053-z . - DOI
    1. Ngo, A, Gandhi, P, Miller, WG. Frequency that laboratory tests influence medical decisions. J Appl Lab Med 2017;1:410–4. https://doi.org/10.1373/jalm.2016.021634 . - DOI
    1. Rohr, UP, Binder, C, Dieterle, T, Giusti, F, Messina, CG, Toerien, E, et al.. The value of in vitro diagnostic testing in medical practice: a status report. PLoS One 2016;11:e0149856. https://doi.org/10.1371/journal.pone.0149856 . - DOI
    1. OpenAI . Chatbot generative pre-trained transformer, ChatGPT. Available from: https://openai.com/blog/chatgpt [Accessed 6 Apr 2023].
    1. Kung, TH, Cheatham, M, Medenilla, A, Sillos, C, Leon, LD, Elepaño, C, et al.. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. Dagan A, editor. PLoS Digit Health 2023;2:e0000198. https://doi.org/10.1371/journal.pdig.0000198 . - DOI