Open E-survey on the Use and Perception of Chatbots in Vascular Surgery

Abstract

Objective: Large language models and artificial intelligence (AI) based chatbots have brought new insights in healthcare, but they also raise major concerns. Their applications in vascular surgery have scarcely been investigated to date. This international survey aimed to evaluate the perceptions and feedback from vascular surgeons on the use of AI chatbots in vascular surgery.

Methods: This international open e-survey comprised 50 items that covered participant characteristics, their perceptions on the use of AI chatbots in vascular surgery, and their user experience. The study was designed in accordance with the Checklist for reporting Results of Internet E-Surveys and was critically reviewed and approved by international members of the European Vascular Research Collaborative (EVRC) prior to distribution. Participation was open to self reported health professionals specialised (or specialising) in vascular surgery, including residents or fellows.

Results: Of the 342 individuals who visited the survey page, 318 (93%) agreed to participate; 262 (82.4%) finished the survey and were included in the analysis. Most were consultants or attending physicians (64.1%), most declared not having any training or education related to AI in healthcare (221; 84.4%), and 198 (75.6%) rated their knowledge about the abilities of AI chatbots between average to very poor. Interestingly, 95 participants (36.3%) found that AI chatbots were very useful or somewhat useful in clinical practice at this stage and 229 (87.4%) agreed that they should be systematically validated prior to being used. Eighty participants (30.5%) had specifically tested it for questions related to clinical practice and 59 (73.8%) of them experienced issues or limitations.

Conclusion: This international survey provides an overview of perceptions of AI chatbots by vascular surgeons and highlights the need to improve knowledge and training of health professionals to better evaluate, define, and implement their use in vascular surgery.

Keywords: Artificial intelligence; Chatbot; Large language model; Natural language processing; Survey; Virtual assistant.