Risks and benefits of ChatGPT in informing patients and families with rare kidney diseases: an explorative assessment by the European Rare Kidney Disease Reference Network (ERKNet)

Abstract

Background: Rare diseases affect fewer than 1 in 2000 individuals, but approximately 150 rare kidney diseases account for about 10% of the chronic kidney disease (CKD) population, impacting millions across Europe and globally. The scarcity of medical experts for these conditions results in an unmet need for accurate and helpful patient information. Large language models like ChatGPT may offer a technological solution to assist medical professionals in educating patients and improving doctor-patient communication. We hypothesized that ChatGPT could provide accurate responses to frequently asked basic questions from patients with rare kidney diseases.

Methods: Medical professionals and members of European Patient Advocacy Groups (ePAGs) affiliated with the European Rare Kidney Disease Reference Network (ERKNet) simulated patient-ChatGPT interactions using a Microsoft forms questionnaire and ChatGPT 3.5 and 4.0. Participants selected any rare kidney disease for a structured conversation with ChatGPT 3.5 or 4.0. Responses were evaluated for accuracy and helpfulness.

Results: Forty-six ERKNet experts and 12 ePAGs from 13 European countries participated in this study. ChatGPT provided scientifically accurate and helpful information on 28 randomly selected rare kidney diseases, including prognostic information and genetic testing guidance. Participants expressed neutral positions regarding ChatGPT's recommendations on alternative treatments, second opinions, and other information sources. While ChatGPT generally was perceived as helpful and empathetic, concerns about patient safety persisted.

Conclusions: ChatGPT exhibited substantial potential in addressing patient inquiries regarding rare kidney diseases in a real-world context. While it demonstrated resilience against misinformation in this application, careful human oversight remains essential and indispensable.

Keywords: AI; Artificial intelligence; ChatGPT; Genetic testing; Large language model; Patient advocacy; Patient education; Rare disease; Rare kidney disease.

A higher resolution version of the Graphical abstract is available as Supplementary information

Fig. 1

Twenty-eight rare kidney diseases selected by 54 ERKNet experts and ePAGs. ADPKD, autosomal dominant polycystic kidney disease; ADTKD, autosomal dominant tubulointerstitial kidney disease; aHUS, atypical hemolytic uremic syndrome; APRTD, adenine phosphoribosyltransferase deficiency; ARPKD, autosomal recessive polycystic kidney disease; CF, cystic fibrosis; FHHNC, familial primary hypomagnesemia with hypercalciuria and nephrocalcinosis; FMF, familial mediterranean fever; MSpK, medullary sponge kidney; NDI, nephrogenic diabetes insipidus; NPHP, nephronophthisis; PHA, pseudohypoaldosteronism; PUV, posterior urethral valves; TMA, thrombotic microangiopathy; XLH, X-linked hypophosphatemia