Validity and reliability of a virtual reality upper gastrointestinal simulator and cross validation using structured assessment of individual performance with video playback

Abstract

Background: This study aims to evaluate the ability of an upper gastrointestinal virtual reality simulator to assess skills in endoscopy, and to validate its metrics using a video-endoscopic (VES) technique.

Methods: The 32 participants in this study were requested to undertake two cases on the simulator (Simbionix, Israel). Each module was repeated twice. The simulator's metrics of performance were used for analysis. two blinded observers rated performance watching the simulator's playback feature.

Results: There were 11 novices (group 1), 11 trainees with intermediate experience (10-50 procedures, group 2), and 10 experienced endoscopists (>200 procedures, group 3). There was a significant difference in the total time required to perform the procedure (p < 0.001), percentage of mucosa visualized (p < 0.001), percentage of pathologies visualized (p < 0.001), and number of inappropriate retroflexions (p = 0.015) across the three groups. The reliability of assessment on the simulator was greater than 0.80 for all parameters. The VES assessment also was able to discriminate performance across the groups (p < 0.001). There was a significant correlation between the VES score and the percentage of mucosa visualized (rho = 0.60; p < 0.001).

Conclusions: The upper gastrointestinal simulator may be a useful tool for determining whether a trainee has achieved a desired level of competence in endoscopy. The next step will be to validate the VES score in real procedures.