Effectiveness of the Medical Chatbot PROSCA to Inform Patients About Prostate Cancer: Results of a Randomized Controlled Trial

Abstract

Background and objective: Artificial intelligence (AI)-powered conversational agents are increasingly finding application in health care, as these can provide patient education at any time. However, their effectiveness in medical settings remains largely unexplored. This study aimed to assess the impact of the chatbot "PROState cancer Conversational Agent" (PROSCA), which was trained to provide validated support from diagnostic tests to treatment options for men facing prostate cancer (PC) diagnosis.

Methods: The chatbot PROSCA, developed by urologists at Heidelberg University Hospital and SAP SE, was evaluated through a randomized controlled trial (RCT). Patients were assigned to either the chatbot group, receiving additional access to PROSCA alongside standard information by urologists, or the control group (1:1), receiving standard information. A total of 112 men were included, of whom 103 gave feedback at study completion.

Key findings and limitations: Over time, patients' information needs decreased significantly more in the chatbot group than in the control group (p = 0.035). In the chatbot group, 43/54 men (79.6%) used PROSCA, and all of them found it easy to use. Of the men, 71.4% agreed that the chatbot improved their informedness about PC and 90.7% would like to use PROSCA again. Limitations are study sample size, single-center design, and specific clinical application.

Conclusions and clinical implications: With the introduction of the PROSCA chatbot, we created and evaluated an innovative, evidence-based AI health information tool as an additional source of information for PC. Our RCT results showed significant benefits of the chatbot in reducing patients' information needs and enhancing their understanding of PC. This easy-to-use AI tool provides accurate, timely, and accessible support, demonstrating its value in the PC diagnosis process. Future steps include further customization of the chatbot's responses and integration with the existing health care systems to maximize its impact on patient outcomes.

Patient summary: This study evaluated an artificial intelligence-powered chatbot-PROSCA, a digital tool designed to support men facing prostate cancer diagnosis by providing validated information from diagnosis to treatment. Results showed that patients who used the chatbot as an additional tool felt better informed than those who received standard information from urologists. The majority of users appreciated the ease of use of the chatbot and expressed a desire to use it again; this suggests that PROSCA could be a valuable resource to improve patient understanding in prostate cancer diagnosis.

Keywords: Artificial intelligence; Chatbot; Early detection; Large language model; Natural language processing; Prostate cancer; Randomized controlled trial.

Fig. 1

PROSCA’s chat design and exemplary conversation. Chat interface design, exemplary text message input, and corresponding text message as well as image and button choice output are shown. The chatbot has been translated into English; the original chatbot pages can be found in Supplementary Figure 1. PROSCA = PROState cancer Conversational Agent.

Fig. 2

Study flowchart according to CONSORT. After inclusion criteria check, patients were randomized into the chatbot and control groups. The chatbot group was further categorized into actual chatbot users for a feasibility and usability analysis.

Fig. 3

Frequency of intent recognition by PROSCA—frequency of intents that were triggered more than 15 times by the natural language processing model during the study period. The intents were triggered by a user’s input and led to an action of the chatbot such as a text reply. The intent description used in this figure is a summary of the theme or category that the intent was created for. MRI = magnetic resonance imaging; PC = prostate cancer; PROSCA = PROState cancer Conversational Agent.

Fig. 4

Evaluation of patient satisfaction with the chatbot—patients’ evaluation regarding whether (A) they would like to reuse the chatbot PROSCA during another hospital visit and (B) they recommend the general implementation of the chatbot PROSCA in clinical routine. PROSCA = PROState cancer Conversational Agent.