Objective psychomotor skills assessment of experienced and novice flexible endoscopists with a virtual reality simulator

Abstract

The objective of this study was to determine whether the GI Mentor II virtual reality simulator can distinguish the psychomotor skills of intermediately experienced endoscopists from those of novices, and do so with a high level of consistency and reliability. A total of five intermediate and nine novice endoscopists were evaluated using the EndoBubble abstract psychomotor task. Each subject performed three repetitions of the task. Performance and error data were recorded for each trial. The intermediate group performed better than the novice group in each trial. The differences were significant in trial 1 for balloons popped (P=.001), completion time (P=.04), and errors (P=.03). Trial 2 showed significance only for balloons popped (P=.002). Trial 3 showed significance for balloons popped (P=.004) and errors (P=.008). The novice group showed significant improvement between trials 1 and 3 (P<0.05). No improvement was noted in the intermediate group. Measures of consistency and reliability were greater than 0.8 in both groups with the exception of novice completion time where test-retest reliability was 0.74. The GI Mentor II simulator can distinguish between novice and intermediate endoscopists. The simulator assesses skills with levels of consistency and reliability required for high-stakes assessment.