Enhancing diagnostics: ChatGPT-4 performance in ulcerative colitis endoscopic assessment

Abstract

Background and study aims: The Mayo Endoscopic Subscore (MES) is widely utilized for assessing mucosal activity in ulcerative colitis (UC). Artificial intelligence has emerged as a promising tool for enhancing diagnostic precision and addressing interobserver variability. This study evaluated the diagnostic accuracy of ChatGPT-4, a multimodal large language model, in identifying and grading endoscopic images of UC patients using the MES.

Patients and methods: Real-world endoscopic images of UC patients were reviewed by an expert consensus board. Each image was graded based on the MES. Only images that were uniformly graded were subsequently provided to three inflammatory bowel disease (IBD) specialists and ChatGPT-4. Severity gradings of the IBD specialists and ChatGPT-4 were compared with assessments made by the expert consensus board.

Results: Thirty of 50 images were graded with complete agreement among the experts. Compared with the consensus board, ChatGPT-4 gradings had a mean accuracy rate of 78.9% whereas the mean accuracy rate for the IBD specialists was 81.1%. Between the two groups, there was no statistically significant difference in mean accuracy rates ( P = 0.71) and a high degree of reliability was found.

Conclusions: ChatGPT-4 has the potential to assess mucosal inflammation severity from endoscopic images of UC patients, without prior configuration or fine-tuning. Performance rates were comparable to those of IBD specialists.

Keywords: Diagnosis and imaging (inc chromoendoscopy, NBI, iSCAN, FICE, CLE...); Endoscopy Lower GI Tract; Image and data processing, documentatiton; Inflammatory bowel disease; Quality and logistical aspects.

Fig. 1

ChatGPT-4 responses of MES assessments.

Fig. 2

Mean accuracy rates between the IBD specialists and ChatGPT-4.