Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2024 Mar;90(3):662-674.
doi: 10.1111/bcp.15963. Epub 2023 Dec 3.

Clinical decision-making in benzodiazepine deprescribing by healthcare providers vs. AI-assisted approach

Iva Bužančić  1   2 , Dora Belec  1 , Margita Držaić  1   2 , Ingrid Kummer  3 , Jovana Brkić  3   4 , Daniela Fialová  3   5 , Maja Ortner Hadžiabdić  1
Affiliations
Free article

Clinical decision-making in benzodiazepine deprescribing by healthcare providers vs. AI-assisted approach

Iva Bužančić et al. Br J Clin Pharmacol. 2024 Mar.
Free article

Abstract

Aims: The aim of this study was to compare the clinical decision-making for benzodiazepine deprescribing between a healthcare provider (HCP) and an artificial intelligence (AI) chatbot GPT4 (ChatGPT-4).

Methods: We analysed real-world data from a Croatian cohort of community-dwelling benzodiazepine patients (n = 154) within the EuroAgeism H2020 ESR 7 project. HCPs evaluated the data using pre-established deprescribing criteria to assess benzodiazepine discontinuation potential. The research team devised and tested AI prompts to ensure consistency with HCP judgements. An independent researcher employed ChatGPT-4 with predetermined prompts to simulate clinical decisions for each patient case. Data derived from human-HCP and ChatGPT-4 decisions were compared for agreement rates and Cohen's kappa.

Results: Both HPC and ChatGPT identified patients for benzodiazepine deprescribing (96.1% and 89.6%, respectively), showing an agreement rate of 95% (κ = .200, P = .012). Agreement on four deprescribing criteria ranged from 74.7% to 91.3% (lack of indication κ = .352, P < .001; prolonged use κ = .088, P = .280; safety concerns κ = .123, P = .006; incorrect dosage κ = .264, P = .001). Important limitations of GPT-4 responses were identified, including 22.1% ambiguous outputs, generic answers and inaccuracies, posing inappropriate decision-making risks.

Conclusions: While AI-HCP agreement is substantial, sole AI reliance poses a risk for unsuitable clinical decision-making. This study's findings reveal both strengths and areas for enhancement of ChatGPT-4 in the deprescribing recommendations within a real-world sample. Our study underscores the need for additional research on chatbot functionality in patient therapy decision-making, further fostering the advancement of AI for optimal performance.

Keywords: ChatGPT-4; artificial intelligence (AI); benzodiazepines; chatbot; deprescribing.

PubMed Disclaimer

References

REFERENCES

    1. Liu S, Wright APA, Patterson BL, et al. Using AI-generated suggestions from ChatGPT to optimize clinical decision support. J Am Med Informatics Assoc. 2023;30(7):1237-1245. doi:10.1093/jamia/ocad072
    1. Giordano C, Brennan M, Mohamed B, Rashidi P, Modave F, Tighe P. Accessing artificial intelligence for clinical decision-making. Front Digit Heal. 2021;25(3):645232.
    1. Markota M, Rummans TA, Bostwick JM, Lapid MI. Benzodiazepine use in older adults: dangers, management, and alternative therapies. Mayo Clin Proc. 2016;91(11):1632-1639. doi:10.1016/j.mayocp.2016.07.024
    1. Gupta A, Bhattacharya G, Balaram K, Tampi D, Tampi RR. Benzodiazepine use among older adults. Neurodegener Dis Manag. 2021;11(1):5-8. doi:10.2217/nmt-2020-0056
    1. Reeve E, Gnjidic D, Long J, Hilmer S. A systematic review of the emerging definition of ‘deprescribing’ with network analysis: implications for future research and clinical practice. Br J Clin Pharmacol. 2015;80(6):1254-1268. doi:10.1111/bcp.12732

Publication types

Substances

LinkOut - more resources