Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Jul 1:13:a26318030.
doi: 10.1055/a-2631-8030. eCollection 2025.

Diagnostic performance and agreement of auditors for evaluation of computer-aided optical polyp diagnosis: Prospective study

Felix Huang  1 , Thea Iulia Dimbu  2 , Douglas K Rex  3 , Heiko Pohl  4 , Cesare Hassan  5   6 , Roupen Djinbachian  1 , Victoire Michal  1 , Dong Hyun Kim  7 , Bilal Amani  1 , Nahlah Haddouch  2 , Sofie Fournier  7 , Daniel von Renteln  1
Affiliations

Diagnostic performance and agreement of auditors for evaluation of computer-aided optical polyp diagnosis: Prospective study

Felix Huang et al. Endosc Int Open. .

Abstract

Background and study aims: Guidelines recommend independent auditing of diagnostic performance for clinical implementation of computer-aided optical polyp diagnosis (CADx). This study evaluated diagnostic performance and interobserver agreement of auditors and offered guidance on conducting CADx audits.

Methods: Images and videos of all ≤ 5-mm polyps from a large, prospective study with systematic activation of CADx were audited by three expert endoscopists. Experts performed independent, blinded diagnostic review including documentation of confidence level. The primary outcome was sensitivity of audit by three experts for high-confidence adenomas compared with pathology. Secondary outcomes included number of reviewers for optimal CADx auditing and interobserver agreement.

Results: Four hundred eighty-seven diminutive polyps were audited (510 patients). Sensitivity was 99.4% (95% confidence interval [CI] 96.0-100) using three experts (Strategy A); 88.7% (95% CI 84.1-92.1) using two experts and one referee (Strategy B); 99% (95% CI 96-99.8), 98.8% (95% CI 95.4-99.8), and 99.4% (95% CI 96.3-100) using two-expert combinations (Strategy C); and 98.2% (95% CI 95.1-99.4), 97.3% (95% CI 94.0-98.9), and 88.9% (95% CI 83.6-92.7) for each expert individually (Strategy D). Among 266 pathology-based adenomas, Strategy A evaluated 160 polyps versus 196, 172, and 170 in Strategy C; and 220, 223, and 207 in Strategy D. Strategy B evaluated all 266 adenomas. Overall interobserver agreement was moderate (kappa 0.52), but very high for high-confidence adenomas (kappa 0.89).

Conclusions: Expert audit for evaluating CADx resulted in high sensitivity and interobserver agreement for high-confidence adenomas. Audit by two experts, with a third expert for arbitration, permitted audit of all polyps and effective assessment of CADx within clinical studies.

Keywords: Endoscopy Lower GI Tract; GI Pathology; Polyps / adenomas / ...; Quality and logistical aspects; Quality management.

PubMed Disclaimer

Conflict of interest statement

Conflict of Interest Daniel von Renteln has received research funding from ERBE Elektromedizin GmbH, Ventage, Pendopharm, Satisfai, Fujifilm, and Pentax, and has received consultant or speaker fees from Boston Scientific Inc., ERBE Elektromedizin GmbH, Medtronic, and Pendopharm. Roupen Djinbachian has received speaker fees from Fujifilm. The remaining authors declare that they have no conflicts of interest.

Figures

Fig. 1
Fig. 1
Study flow chart.
Fig. 2
Fig. 2
Diagnostic characteristics of each audit strategy (%, 95% confidence interval [CI]). For each performance metric and audit strategy, the number of polyps included in the denominator is indicated at the bottom of the panels.
Fig. 3
Fig. 3
Expert interobserver agreement according to polyp histology and confidence level. Dark blue: no agreement; dark green: near perfect agreement.
See this image and copyright information in PMC

References

    1. Djinbachian R, Rex DK, von Renteln D. Optical polyp diagnosis in the era or artificial intelligence. Am J Gastroenterol. 2024;120:1268–1274. doi: 10.14309/ajg.0000000000002664. - DOI - PubMed
    1. Kessler WR, Imperiale TF, Klein RW et al. A quantitative assessment of the risks and cost savings of forgoing histologic examination of diminutive polyps. Endoscopy. 2011;43:683–691. doi: 10.1055/s-0030-1256381. - DOI - PubMed
    1. Rex DK, Kahi C, O’Brien M et al. The American Society for Gastrointestinal Endoscopy PIVI (Preservation and Incorporation of Valuable Endoscopic Innovations) on real-time endoscopic assessment of the histology of diminutive colorectal polyps. Gastrointest Endosc. 2011;73:419–422. doi: 10.1016/j.gie.2011.01.023. - DOI - PubMed
    1. Ahmad A, Moorghen M, Wilson A et al. Implementation of optical diagnosis with a "resect and discard" strategy in clinical practice: DISCARD3 study. Gastrointest Endosc. 2022;96:1021–103200. - PubMed
    1. Barua I, Wieszczy P, Kudo SE et al. Real-time artificial intelligence-based optical diagnosis of neoplastic polyps during colonoscopy. NEJM Evid. 2022;1EVIDoa2200003 - PubMed

LinkOut - more resources