ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis

Elisabetta Maida^#¹, Marcello Moccia^#^{2

3}, Raffaele Palladino^{4

5}, Giovanna Borriello⁶, Giuseppina Affinito⁴, Marinella Clerico⁷, Anna Maria Repice⁸, Alessia Di Sapio⁹, Rosa Iodice¹⁰, Antonio Luca Spiezia³, Maddalena Sparaco¹, Giuseppina Miele¹, Floriana Bile¹, Cristiano Scandurra¹⁰, Diana Ferraro^{11

12}, Maria Laura Stromillo¹³, Renato Docimo¹⁴, Antonio De Martino¹⁵, Luca Mancinelli¹⁶, Gianmarco Abbadessa^{1

17}, Krzysztof Smolik¹², Lorenzo Lorusso¹⁸, Maurizio Leone¹⁹, Elisa Leveraro^{20

21}, Francesca Lauro¹⁰, Francesca Trojsi¹, Lidia Mislin Streito⁷, Francesca Gabriele²², Fabiana Marinelli²³, Antonio Ianniello⁶, Federica De Santis²⁴, Matteo Foschi^{22

25}, Nicola De Stefano¹³, Vincenzo Brescia Morra¹⁰, Alvino Bisecco¹⁴, Giancarlo Coghe²⁶, Eleonora Cocco²⁶, Michele Romoli¹⁶, Francesco Corea²⁷, Letizia Leocani^{28

29}, Jessica Frau²⁶, Simona Sacco²², Matilde Inglese^{20

21}, Antonio Carotenuto¹⁰, Roberta Lanzillo¹⁰, Alessandro Padovani^{30

31}, Maria Triassi⁴, Simona Bonavita³², Luigi Lavorgna¹; Digital Technologies, Web, Social Media Study Group of the Italian Society of Neurology (SIN)

Affiliations

¹ Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Via Pansini 5, 80131, Naples, Italy.
² Department of Molecular Medicine and Medical Biotechnology, Federico II University of Naples, Naples, Italy.
³ Multiple Sclerosis Unit, Policlinico Federico II University Hospital, Naples, Italy.
⁴ Department of Public Health, University "Federico II" of Naples, Naples, Italy.
⁵ Department of Primary Care and Public Health, Imperial College of London, London, UK.
⁶ Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy.
⁷ Dipartimento di Scienze Cliniche e Biologiche, Università Di Torino, Turin, Italy.
⁸ Department of Neurology 2 and Tuscan Region Multiple Sclerosis Referral Centre, Careggi University Hospital, Florence, Italy.
⁹ Department of Neurology and Multiple Sclerosis Regional Referral Centre, AOU San Luigi Gonzaga, Orbassano, Turin, Italy.
¹⁰ Department of Neurosciences, Reproductive Sciences and Odontostomatology, University of Naples Federico II, Naples, Italy.
¹¹ Department of Neuroscience, Azienda Ospedaliero-Universitaria di Modena, Modena, Emilia-Romagna, Italy.
¹² Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy.
¹³ Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy.
¹⁴ Multiple Sclerosis Center, Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Naples, Italy.
¹⁵ Institute of Neurology, University Magna Graecia of Catanzaro, Catanzaro, Italy.
¹⁶ Neurology Unit, Bufalini Hospital, Local Health Agency of Romagna, Cesena, Italy.
¹⁷ Department of Brain Sciences, Imperial College London, London, W120BZ, UK.
¹⁸ Neurology Unit-Neuroscience Department A.S.S.T.Lecco, Merate Hospital, 23807, Merate, Italy.
¹⁹ Neurology Unit, Fondazione IRCCS Casa Sollievo della Sofferenza, 71013, San Giovanni Rotondo, Italy.
²⁰ Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health (DiNOGMI), University of Genoa, Genoa, Italy.
²¹ Department of Neurology, IRCSS Ospedale Policlinico San Martino, Genoa, Italy.
²² Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, L'Aquila, Italy.
²³ Neurology Unit, Multiple Sclerosis Center, Fabrizio Spaziani Hospital, Frosinone, Italy.
²⁴ Department of Neurology and Stroke Unit of Avezzano-Sulmona, ASL 1 Avezzano-Sulmona-L'Aquila, L'Aquila, Italy.
²⁵ Department of Neuroscience, Multiple Sclerosis Center, S. Maria delle Croci Hospital, AUSL Romagna, Ravenna, Italy.
²⁶ Department of Medical Sciences and Public Health, Multiple Sclerosis Center, Binaghi Hospital, ASL Cagliari, University of Cagliari, Cagliari, Italy.
²⁷ Dipartimento di Neurologia, Ospedale di Foligno, Azienda USL Umbria 2, Terni, Italy.
²⁸ Vita-Salute San Raffaele University, Milan, Italy.
²⁹ Experimental Neurophysiology Unit, Institute of Experimental Neurology-INSPE, IRCCS Scientific Institute San Raffaele, Milan, Italy.
³⁰ Unit of Neurology, Azienda Socio-Sanitaria Territoriale Spedali Civili, Brescia, Italy.
³¹ Department of Clinical and Experimental Sciences, University of Brescia, Brescia, Italy.
³² Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Via Pansini 5, 80131, Naples, Italy. simona.bonavita@unicampania.it.

^# Contributed equally.

PMID: 38568227
PMCID: PMC11233331
DOI: 10.1007/s00415-024-12328-x

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis

Elisabetta Maida et al. J Neurol. 2024 Jul.

. 2024 Jul;271(7):4057-4066.

doi: 10.1007/s00415-024-12328-x. Epub 2024 Apr 3.

Authors

Affiliations

¹ Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Via Pansini 5, 80131, Naples, Italy.
² Department of Molecular Medicine and Medical Biotechnology, Federico II University of Naples, Naples, Italy.
³ Multiple Sclerosis Unit, Policlinico Federico II University Hospital, Naples, Italy.
⁴ Department of Public Health, University "Federico II" of Naples, Naples, Italy.
⁵ Department of Primary Care and Public Health, Imperial College of London, London, UK.
⁶ Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy.
⁷ Dipartimento di Scienze Cliniche e Biologiche, Università Di Torino, Turin, Italy.
⁸ Department of Neurology 2 and Tuscan Region Multiple Sclerosis Referral Centre, Careggi University Hospital, Florence, Italy.
⁹ Department of Neurology and Multiple Sclerosis Regional Referral Centre, AOU San Luigi Gonzaga, Orbassano, Turin, Italy.
¹⁰ Department of Neurosciences, Reproductive Sciences and Odontostomatology, University of Naples Federico II, Naples, Italy.
¹¹ Department of Neuroscience, Azienda Ospedaliero-Universitaria di Modena, Modena, Emilia-Romagna, Italy.
¹² Department of Biomedical, Metabolic and Neural Sciences, University of Modena and Reggio Emilia, Modena, Italy.
¹³ Department of Medicine, Surgery and Neuroscience, University of Siena, Siena, Italy.
¹⁴ Multiple Sclerosis Center, Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Naples, Italy.
¹⁵ Institute of Neurology, University Magna Graecia of Catanzaro, Catanzaro, Italy.
¹⁶ Neurology Unit, Bufalini Hospital, Local Health Agency of Romagna, Cesena, Italy.
¹⁷ Department of Brain Sciences, Imperial College London, London, W120BZ, UK.
¹⁸ Neurology Unit-Neuroscience Department A.S.S.T.Lecco, Merate Hospital, 23807, Merate, Italy.
¹⁹ Neurology Unit, Fondazione IRCCS Casa Sollievo della Sofferenza, 71013, San Giovanni Rotondo, Italy.
²⁰ Department of Neuroscience, Rehabilitation, Ophthalmology, Genetics, Maternal and Child Health (DiNOGMI), University of Genoa, Genoa, Italy.
²¹ Department of Neurology, IRCSS Ospedale Policlinico San Martino, Genoa, Italy.
²² Department of Biotechnological and Applied Clinical Sciences, University of L'Aquila, L'Aquila, Italy.
²³ Neurology Unit, Multiple Sclerosis Center, Fabrizio Spaziani Hospital, Frosinone, Italy.
²⁴ Department of Neurology and Stroke Unit of Avezzano-Sulmona, ASL 1 Avezzano-Sulmona-L'Aquila, L'Aquila, Italy.
²⁵ Department of Neuroscience, Multiple Sclerosis Center, S. Maria delle Croci Hospital, AUSL Romagna, Ravenna, Italy.
²⁶ Department of Medical Sciences and Public Health, Multiple Sclerosis Center, Binaghi Hospital, ASL Cagliari, University of Cagliari, Cagliari, Italy.
²⁷ Dipartimento di Neurologia, Ospedale di Foligno, Azienda USL Umbria 2, Terni, Italy.
²⁸ Vita-Salute San Raffaele University, Milan, Italy.
²⁹ Experimental Neurophysiology Unit, Institute of Experimental Neurology-INSPE, IRCCS Scientific Institute San Raffaele, Milan, Italy.
³⁰ Unit of Neurology, Azienda Socio-Sanitaria Territoriale Spedali Civili, Brescia, Italy.
³¹ Department of Clinical and Experimental Sciences, University of Brescia, Brescia, Italy.
³² Department of Advanced Medical and Surgical Sciences, University of Campania "Luigi Vanvitelli", Via Pansini 5, 80131, Naples, Italy. simona.bonavita@unicampania.it.

^# Contributed equally.

PMID: 38568227
PMCID: PMC11233331
DOI: 10.1007/s00415-024-12328-x

Abstract

Background: ChatGPT is an open-source natural language processing software that replies to users' queries. We conducted a cross-sectional study to assess people living with Multiple Sclerosis' (PwMS) preferences, satisfaction, and empathy toward two alternate responses to four frequently-asked questions, one authored by a group of neurologists, the other by ChatGPT.

Methods: An online form was sent through digital communication platforms. PwMS were blind to the author of each response and were asked to express their preference for each alternate response to the four questions. The overall satisfaction was assessed using a Likert scale (1-5); the Consultation and Relational Empathy scale was employed to assess perceived empathy.

Results: We included 1133 PwMS (age, 45.26 ± 11.50 years; females, 68.49%). ChatGPT's responses showed significantly higher empathy scores (Coeff = 1.38; 95% CI = 0.65, 2.11; p > z < 0.01), when compared with neurologists' responses. No association was found between ChatGPT' responses and mean satisfaction (Coeff = 0.03; 95% CI = - 0.01, 0.07; p = 0.157). College graduate, when compared with high school education responder, had significantly lower likelihood to prefer ChatGPT response (IRR = 0.87; 95% CI = 0.79, 0.95; p < 0.01).

Conclusions: ChatGPT-authored responses provided higher empathy than neurologists. Although AI holds potential, physicians should prepare to interact with increasingly digitized patients and guide them on responsible AI use. Future development should consider tailoring AIs' responses to individual characteristics. Within the progressive digitalization of the population, ChatGPT could emerge as a helpful support in healthcare management rather than an alternative.

Keywords: Artificial intelligence; Large language model; Machine learning; Multiple sclerosis.

PubMed Disclaimer

Conflict of interest statement

The authors declare that they have no conflict of interest.

References

1. Jiang F, Jiang Y, Zhi H, Dong Y, Li H, Ma S, Wang Y, Dong Q, Shen H, Wang Y. Artificial intelligence in healthcare: past, present and future. Stroke Vasc Neurol. 2017;2(4):230–243. doi: 10.1136/svn-2017-000101. - DOI - PMC - PubMed
1. Ortiz M, Mallen V, Boquete L, Sánchez-Morla EM, Cordón B, Vilades E, Dongil-Moreno FJ, Miguel-Jiménez JM, Garcia-Martin E. Diagnosis of multiple sclerosis using optical coherence tomography supported by artificial intelligence. Mult Scler Relat Disord. 2023;74:104725. doi: 10.1016/j.msard.2023.104725. - DOI - PubMed
1. Afzal HMR, Luo S, Ramadan S, Lechner-Scott J. The emerging role of artificial intelligence in multiple sclerosis imaging. Mult Scler. 2022;28(6):849–858. doi: 10.1177/1352458520966298. - DOI - PubMed
1. Zivadinov R, Bergsland N, Jakimovski D, Weinstock-Guttman B, Benedict RHB, Riolo J, Silva D, Dwyer MG. DeepGRAI registry study group. Thalamic atrophy measured by artificial intelligence in a multicentre clinical routine real-word study is associated with disability progression. J Neurol Neurosurg Psychiatry jnnp. 2022 doi: 10.1136/jnnp-2022-329333. - DOI - PubMed
1. ChatGPT. https://openai.com/blog/chatgpt. Accessed Dec 2023

MeSH terms

Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions
Actions

LinkOut - more resources

Full Text Sources
- PubMed Central
- Springer
Medical
- MedlinePlus Health Information

Save citation to file

Email citation

Add to Collections

Add to My Bibliography

Your saved search

Create a file for external citation management software

Your RSS Feed

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis

Affiliations

ChatGPT vs. neurologists: a cross-sectional study investigating preference, satisfaction ratings and perceived empathy in responses among people living with multiple sclerosis

Authors

Affiliations

Abstract

Conflict of interest statement

References

MeSH terms

LinkOut - more resources

Full Text Sources

Medical