Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2022 Oct;10(8):817-826.
doi: 10.1002/ueg2.12285. Epub 2022 Aug 19.

Endoscopists performance in optical diagnosis of colorectal polyps in artificial intelligence studies

Silvia Pecere  1   2 , Giulio Antonelli  3   4 , Mario Dinis-Ribeiro  5 , Yuichi Mori  6   7 , Cesare Hassan  8   9 , Lorenzo Fuccio  10 , Raf Bisschops  11 , Guido Costamagna  1   2 , Eun Hyo Jin  12 , Dongheon Lee  13 , Masashi Misawa  7 , Helmut Messmann  14 , Federico Iacopini  4 , Lucio Petruzziello  1   2 , Alessandro Repici  8 , Yutaka Saito  15 , Prateek Sharma  16 , Masayoshi Yamada  15 , Cristiano Spada  2   17 , Leonardo Frazzoni  10
Affiliations
Review

Endoscopists performance in optical diagnosis of colorectal polyps in artificial intelligence studies

Silvia Pecere et al. United European Gastroenterol J. 2022 Oct.

Abstract

Widespread adoption of optical diagnosis of colorectal neoplasia is prevented by suboptimal endoscopist performance and lack of standardized training and competence evaluation. We aimed to assess diagnostic accuracy of endoscopists in optical diagnosis of colorectal neoplasia in the framework of artificial intelligence (AI) validation studies. Literature searches of databases (PubMed/MEDLINE, EMBASE, Scopus) up to April 2022 were performed to identify articles evaluating accuracy of individual endoscopists in performing optical diagnosis of colorectal neoplasia within studies validating AI against a histologically verified ground-truth. The main outcomes were endoscopists' pooled sensitivity, specificity, positive and negative predictive value (PPV/NPV), positive and negative likelihood ratio (LR) and area under the curve (AUC for sROC) for predicting adenomas versus non-adenomas. Six studies with 67 endoscopists and 2085 (IQR: 115-243,5) patients were evaluated. Pooled sensitivity and specificity for adenomatous histology was respectively 84.5% (95% CI 80.3%-88%) and 83% (95% CI 79.6%-85.9%), corresponding to a PPV, NPV, LR+, LR- of 89.5% (95% CI 87.1%-91.5%), 75.7% (95% CI 70.1%-80.7%), 5 (95% CI 3.9%-6.2%) and 0.19 (95% CI 0.14%-0.25%). The AUC was 0.82 (CI 0.76-0.90). Expert endoscopists showed a higher sensitivity than non-experts (90.5%, [95% CI 87.6%-92.7%] vs. 75.5%, [95% CI 66.5%-82.7%], p < 0.001), and Eastern endoscopists showed a higher sensitivity than Western (85%, [95% CI 80.5%-88.6%] vs. 75.8%, [95% CI 70.2%-80.6%]). Quality was graded high for 3 studies and low for 3 studies. We show that human accuracy for diagnosis of colorectal neoplasia in the setting of AI studies is suboptimal. Educational interventions could benefit by AI validation settings which seem a feasible framework for competence assessment.

Keywords: artificial intelligence; colonoscopy; endoscopist performance; human factor; polyp characterization; polyp detection.

PubMed Disclaimer

Conflict of interest statement

The authors declare no COI relevant to this paper.

Figures

FIGURE 1
FIGURE 1
Flow‐chart of included studies
FIGURE 2
FIGURE 2
Summary receiver‐operating characteristic curve
FIGURE 3
FIGURE 3
Forest plots for sensitivity and specificity by study
FIGURE 4
FIGURE 4
Forest plots for sensitivity and specificity by experience and country
See this image and copyright information in PMC

Comment in

Similar articles

See all similar articles

Cited by

References

    1. Greuter MJE, de Klerk CM, Meijer GA, Dekker E, Coupé VM. Screening for colorectal cancer with fecal immunochemical testing with and without postpolypectomy surveillance colonoscopy: a cost‐effectiveness analysis. Ann Intern Med. 2017;167:544–54. 10.7326/M16-2891 - DOI - PubMed
    1. Gupta N, Bansal A, Rao D, Early DS, Jonnalagadda S, Wani SB, et al. Prevalence of advanced histological features in diminutive and small colon polyps. Gastrointest Endosc. 2012;75(5):1022–30. 10.1016/j.gie.2012.01.020 - DOI - PubMed
    1. Laish I, Sergeev I, Stein A, Naftali T, Konikoff FM. Risk of metachronous advanced lesions after resection of diminutive and small, non‐advanced adenomas. Clin Res Hepatol Gastroenterol. 2019;43(2):201–7. 10.1016/j.clinre.2018.03.001 - DOI - PubMed
    1. Schachschal G, Sehner S, Choschzick M, Aust D, Brandl L, Vieth M, et al. Impact of reassessment of colonic hyperplastic polyps by expert GI pathologists. Int J Colorectal Dis. 2016;31(3):675–83. 10.1007/s00384-016-2523-8 - DOI - PubMed
    1. Hassan C, Pickhardt PJ, Rex DK. A resect and discard strategy would improve cost‐effectiveness of colorectal cancer screening. Clin Gastroenterol Hepatol. 2010;8(10):865–9.e3. 10.1016/j.cgh.2010.05.018 - DOI - PubMed

Publication types