Impact of User's Background Knowledge and Polyp Characteristics in Colonoscopy with Computer-Aided Detection

Abstract

Background/aims: We investigated how interactions between humans and computer-aided detection (CADe) systems are influenced by the user's experience and polyp characteristics.

Methods: We developed a CADe system using YOLOv4, trained on 16,996 polyp images from 1,914 patients and 1,800 synthesized sessile serrated lesion (SSL) images. The performance of polyp detection with CADe assistance was evaluated using a computerized test module. Eighteen participants were grouped by colonoscopy experience (nurses, fellows, and experts). The value added by CADe based on the histopathology and detection difficulty of polyps were analyzed.

Results: The area under the curve for CADe was 0.87 (95% confidence interval [CI], 0.83 to 0.91). CADe assistance increased overall polyp detection accuracy from 69.7% to 77.7% (odds ratio [OR], 1.88; 95% CI, 1.69 to 2.09). However, accuracy decreased when CADe inaccurately detected a polyp (OR, 0.72; 95% CI, 0.58 to 0.87). The impact of CADe assistance was most and least prominent in the nurses (OR, 1.97; 95% CI, 1.71 to 2.27) and the experts (OR, 1.42; 95% CI, 1.15 to 1.74), respectively. Participants demonstrated better sensitivity with CADe assistance, achieving 81.7% for adenomas and 92.4% for easy-to-detect polyps, surpassing the standalone CADe performance of 79.7% and 89.8%, respectively. For SSLs and difficult-to-detect polyps, participants' sensitivities with CADe assistance (66.5% and 71.5%, respectively) were below those of standalone CADe (81.1% and 74.4%). Compared to the other two groups (56.1% and 61.7%), the expert group showed sensitivity closest to that of standalone CADe in detecting SSLs (79.7% vs 81.1%, respectively).

Conclusions: CADe assistance boosts polyp detection significantly, but its effectiveness depends on the user's experience, particularly for challenging lesions.

Keywords: Artificial intelligence; Colonoscopy; Polyps.

Fig. 1

Study design. CADe, computer-aided detection; AI, artificial intelligence.

Fig. 2

The effect of CADe on polyp detection according to participating groups. (A) Accuracy. (B) False positive rate. (C) Sensitivity. CADe, computer-aided detection; Group 1, nurses; Group 2, fellows; Group 3, experts.

Fig. 3

The ROC curve evaluating detection performance of the CADe system. ROC, receiver operating characteristic; CADe, computer-aided detection; AUC, area under the curve; Group 1, nurses; Group 2, fellows; Group 3, experts.

Fig. 4

The influence of CADe assistance in polyp detection: an analysis of sensitivity based on polyp characteristics. (A) Adenoma. (B) Sessile serrated lesion. (C) Easy-to-detect polyp. (D) Difficult-to-detect polyp. CADe, computer-aided detection; Group 1, nurses; Group 2, fellows; Group 3, experts.