Using objective robotic automated performance metrics and task-evoked pupillary response to distinguish surgeon expertise

Abstract

Purpose: In this study, we investigate the ability of automated performance metrics (APMs) and task-evoked pupillary response (TEPR), as objective measures of surgeon performance, to distinguish varying levels of surgeon expertise during generic robotic surgical tasks. Additionally, we evaluate the association between APMs and TEPR.

Methods: Participants completed ten tasks on a da Vinci Xi Surgical System (Intuitive Surgical, Inc.), each representing a surgical skill type: EndoWrist^® manipulation, needle targeting, suturing/knot tying, and excision/dissection. Automated performance metrics (instrument motion tracking, EndoWrist^® articulation, and system events data) and TEPR were recorded by a systems data recorder (Intuitive Surgical, Inc.) and Tobii Pro Glasses 2 (Tobii Technologies, Inc.), respectively. The Kruskal-Wallis test determined significant differences between groups of varying expertise. Spearman's rank correlation coefficient measured associations between APMs and TEPR.

Results: Twenty-six participants were stratified by robotic surgical experience: novice (no prior experience; n = 9), intermediate (< 100 cases; n = 9), and experts (≥ 100 cases; n = 8). Several APMs differentiated surgeon experience including task duration (p < 0.01), time active of instruments (p < 0.03), linear velocity of instruments (p < 0.04), and angular velocity of dominant instrument (p < 0.04). Task-evoked pupillary response distinguished surgeon expertise for three out of four task types (p < 0.04). Correlation trends between APMs and TEPR revealed that expert surgeons move more slowly with high cognitive workload (ρ < - 0.60, p < 0.05), while novices move faster under the same cognitive experiences (ρ > 0.66, p < 0.05).

Conclusions: Automated performance metrics and TEPR can distinguish surgeon expertise levels during robotic surgical tasks. Furthermore, under high cognitive workload, there can be a divergence in robotic movement profiles between expertise levels.

Keywords: Automated performance metrics; Robotic surgical training; Surgeon assessment; Task-evoked pupillary response.

Fig. 1

a Task type 1 included 4 ring rollercoasters reflecting EndoWrist® manipulation skills. Participants were instructed to successively guide two rings from one end to another without dropping the ring. b Task type 2 included two needle targeting tasks. c Task type 3 included two suturing/knot tying tasks. Participants were instructed to execute two interrupted knots: either simple knots or figure 8 knots. d Task type 4 reflected two excision/dissection tasks. Participants were instructed to consecutively remove the “lid” and a wedge of a clementine.

Fig. 2

a Intuitive Surgical, Inc.’s system events data recorder, a device that records synchronized endoscopic video and kinematic/system events data through direct physical connection to a da Vinci Surgical System. b Tobii Pro Glasses 2 (Tobii Technology, Inc.), a wearable eye-tracking system.