A Novel Switching of Artificial Intelligence to Generate Simultaneously Multimodal Images to Assess Inflammation and Predict Outcomes in Ulcerative Colitis-(With Video)

Abstract

Objectives: Virtual Chromoendoscopy (VCE) is pivotal for assessing activity and predicting outcomes in Ulcerative Colitis (UC), though interobserver variability and the need for expertise persist. Artificial intelligence (AI) offers standardized VCE-based assessment. This study introduces a novel AI model to detect and simultaneously generate various endoscopic modalities, enhancing AI-driven inflammation assessment and outcome prediction in UC.

Methods: Endoscopic videos in high-definition white-light, iScan2, iScan3, and NBI from UC patients of the international PICaSSO iScan and NBI cohort (302 and 54 patients, respectively) were used to develop a neural network to identify the acquisition modality of each frame and for inter-modality image switching. 2535 frames from 169 videos of the iScan cohort were switched to different modalities and trained a deep-learning model for inflammation assessment. Subsequently, the model was tested on a subset of the iScan and NBI cohorts (72 and 51 videos, respectively). Performance in predicting endoscopic and histological activity and outcomes was evaluated.

Results: The model efficiently classified and converted images across modalities (92% accuracy). Performance in predicting endoscopic and histological remission was excellent, especially with different modalities combined in both iScan (accuracy 81.3% and 89.6%; AUROC 0.92 and 0.89 by UCEIS and PICaSSO, respectively) and the NBI cohort. Moreover, it showed a remarkable ability in predicting clinical outcomes.

Conclusions: Our multimodal "AI-switching" model innovatively detects and transitions between different endoscopic modalities, refining inflammation assessment and outcome prediction in UC by integrating model-derived images.

Keywords: artificial intelligence; ulcerative colitis; virtual chromoendoscopy.