da Vinci Skills Simulator construct validation study: correlation of prior robotic experience with overall score and time score simulator performance

Abstract

Objective: To assess the construct validity of the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA). Ideally, a well-designed simulator should demonstrate construct validity, which is defined in this study as the correlation between robotic surgical experience and performance on the simulator.

Patients and methods: Thirty-nine surgeons (18 [46%] group I [0-20 robotic cases]; 8 [21%] group II [21-150 robotic cases]; and 13 [33%] group III [>150 robotic cases]) were enrolled from September 2010 to December 2010. Participants completed 24 virtual-reality exercises on the da Vinci Skills Simulator. Data on 12 performance metrics were collected by the software. Overall means for score and time across exercises were analyzed.

Results: Overall scores (64.7%/79.1%/87.4%) and time scores (39.1%/58.6%/87.3%) were significantly different among surgeons in groups I-III (P <.001) and demonstrated significant linear relationships (P <.001) for all 24 exercises. Comparisons between the 3 groups using a univariate general linear model (GLM) was used to compare groups I and II and II and III. Groups I and II differed using overall score for 15 exercises and time score for 11 exercises. Groups II and III differed using overall score for 6 exercises and time score for 15 exercises. Mean overall score for 1 exercise displayed significance between both groups I and II and II and III; while using time score, 5 exercises displayed significance between surgeons in groups I and II and II and III.

Conclusion: Initial construct validity analysis revealed that both overall scores and time scores showed a significant linear relationship when comparing the surgeons in groups I, II, and III. Overall score seems to be a stronger indicator for differences between surgeons in groups I and II. Time score seems to be a stronger indicator for differences between surgeons in groups II and III.