Assessing colonoscopic inspection skill using a virtual withdrawal simulation: a preliminary validation of performance metrics

Abstract

Background: The effectiveness of colonoscopy for diagnosing and preventing colon cancer is largely dependent on the ability of endoscopists to fully inspect the colonic mucosa, which they achieve primarily through skilled manipulation of the colonoscope during withdrawal. Performance assessment during live procedures is problematic. However, a virtual withdrawal simulation can help identify and parameterise actions linked to successful inspection, and offer standardised assessments for trainees.

Methods: Eleven experienced endoscopists and 18 endoscopy novices (medical students) completed a mucosal inspection task during three simulated colonoscopic withdrawals. The two groups were compared on 10 performance metrics to preliminarily assess the validity of these measures to describe inspection quality. Four metrics were related to aspects of polyp detection: percentage of polyp markers found; number of polyp markers found per minute; percentage of the mucosal surface illuminated by the colonoscope (≥0.5 s); and percentage of polyp markers illuminated (≥2.5 s) but not identified. A further six metrics described the movement of the colonoscope: withdrawal time; linear distance travelled by the colonoscope tip; total distance travelled by the colonoscope tip; and distance travelled by the colonoscope tip due to movement of the up/down angulation control, movement of the left/right angulation control, and axial shaft rotation.

Results: Statistically significant experienced-novice differences were found for 8 of the 10 performance metrics (p's < .005). Compared with novices, experienced endoscopists inspected more of the mucosa and detected more polyp markers, at a faster rate. Despite completing the withdrawals more quickly than the novices, the experienced endoscopists also moved the colonoscope more in terms of linear distance travelled and overall tip movement, with greater use of both the up/down angulation control and axial shaft rotation. However, the groups did not differ in the number of polyp markers visible on the monitor but not identified, or movement of the left/right angulation control. All metrics that yielded significant group differences had adequate to excellent internal consistency reliability (α = .79 to .90).

Conclusions: These systematic differences confirm the potential of the simulated withdrawal task for evaluating inspection skills and strategies. It may be useful for training, and assessment of trainee competence.

Keywords: Skill assessment; Training; Virtual reality.

Fig. 1

The CSIRO Colonoscopy Simulator

Fig. 2

Simulated colonic mucosa, folds, and a small “polyp marker”

Fig. 3

Mean (±95% CIs) performance for each experience group on measures associated with the detection of polyp markers. Specific measures are: percentage of polyp markers found (a); number of polyp markers found per minute (b); percentage of the mucosal surface illuminated (for 0.5 s or more) by the colonoscope (c); and polyp markers illuminated (for 2.5 s or more) but not identified by the participant, as a percentage of all polyp markers (d). Asterisks indicate statistically significant differences between the groups

Fig. 4

Mean (±95% CIs) performance for each experience group on measures describing the movement of the colonoscope. Specific measures are: withdrawal time (a); linear distance travelled by the colonoscope (b); total distance travelled by the colonoscope tip (c); and distance travelled by the colonoscope tip due to movement of the up/down angulation control (d), movement of the left/right angulation control (e), and axial rotation (f). Asterisks indicate statistically significant differences between the groups