Efficacy of real-time artificial intelligence-aid endoscopic ultrasonography diagnostic system in discriminating gastrointestinal stromal tumors and leiomyomas: a multicenter diagnostic study

Abstract

Background: Gastrointestinal stromal tumors (GISTs) represent the most prevalent type of subepithelial lesions (SELs) with malignant potential. Current imaging tools struggle to differentiate GISTs from leiomyomas. This study aimed to create and assess a real-time artificial intelligence (AI) system using endoscopic ultrasonography (EUS) images to differentiate between GISTs and leiomyomas.

Methods: The AI system underwent development and evaluation using EUS images from 5 endoscopic centers in China between January 2020 and August 2023. EUS images of 1101 participants with SELs were retrospectively collected for AI system development. A cohort of 241 participants with SELs was recruited for external AI system evaluation. Another cohort of 59 participants with SELs was prospectively enrolled to assess the real-time clinical application of the AI system. The AI system's performance was compared to that of endoscopists. This study is registered with Chictr.org.cn, Number ChiCT2000035787.

Findings: The AI system displayed an area under the curve (AUC) of 0.948 (95% CI: 0.921-0.969) for discriminating GISTs and leiomyomas. The AI system's accuracy (ACC), sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) reached 91.7% (95% CI 87.5%-94.6%), 90.3% (95% CI 83.4%-94.5%), 93.0% (95% CI 87.2%-96.3%), 91.9% (95% CI 85.3%-95.7%), and 91.5% (95% CI 85.5%-95.2%), respectively. Moreover, the AI system exhibited excellent performance in diagnosing ≤20 mm SELs (ACC 93.5%, 95% CI 0.900-0.969). In a prospective real-time clinical application trial, the AI system achieved an AUC of 0.865 (95% CI 0.764-0.966) and 0.864 (95% CI 0.762-0.966) for GISTs and leiomyomas diagnosis, respectively, markedly surpassing endoscopists [AUC 0.698 (95% CI 0.562-0.834) for GISTs and AUC 0.695 (95% CI 0.546-0.825) for leiomyomas].

Interpretation: We successfully developed a real-time AI-assisted EUS diagnostic system. The incorporation of the real-time AI system during EUS examinations can assist endoscopists in rapidly and accurately differentiating various types of SELs in clinical practice, facilitating improved diagnostic and therapeutic decision-making.

Funding: Science and Technology Commission Foundation of Shanghai Municipality, Science and Technology Commission Foundation of the Xuhui District, the Interdisciplinary Program of Shanghai Jiao Tong University and the Research Funds of Shanghai Sixth people's Hospital.

Keywords: Artificial intelligence; Endoscopic ultrasound; Gastrointestinal stromal tumors; Gastrointestinal subepithelial lesions; Leiomyomas.

Fig. 1

Flow chart depicting the study process. The illustration outlines participant enrollment in the AI model development cohort and clinical application.

Fig. 2

Comprehensive overview of the AI system model development. Data collection (Left) involves the selection of representative EUS images by endoscopists from 1101 SELs patients. Data cleaning and annotation (Middle) manually segments and delineates the Region of Interest (ROI) of lesions for model training and validation. Model development and clinical evaluation (Right).

Fig. 3

Diagnostic efficiency of the AI system in the internal and external test cohort. ROC curves for GISTs and leiomyomas diagnosis in the internal test set (Left) and the external test set (Right).

Fig. 4

Diagnostic performance of the AI system for different size- and type- SELs in the external evaluation cohort. The accuracy of AI system for diagnosing ≤20 mm SELs is significantly higher (P = 0.031). There is no significant difference in accuracy for different types of SELs (P = 0.448).

Fig. 5

Real-time application assessment of the AI system in a prospective clinical cohort. Performance metrics (including AUC, ACC, SE, SP, PPV, NPV) in diagnosing GISTs and leiomyomas evaluated in a prospective clinical cohort. The AI system demonstrates superior discrimination in clinical settings.