Using Real-time Feedback To Improve Surgical Performance on a Robotic Tissue Dissection Task

Abstract

Background: There is no standard for the feedback that an attending surgeon provides to a training surgeon, which may lead to variable outcomes in teaching cases.

Objective: To create and administer standardized feedback to medical students in an attempt to improve performance and learning.

Design setting and participants: A cohort of 45 medical students was recruited from a single medical school. Participants were randomly assigned to two groups. Both completed two rounds of a robotic surgical dissection task on a da Vinci Xi surgical system. The first round was the baseline assessment. In the second round, one group received feedback and the other served as the control (no feedback).

Outcome measurements and statistical analysis: Video from each round was retrospectively reviewed by four blinded raters and given a total error tally (primary outcome) and a technical skills score (Global Evaluative Assessment of Robotic Surgery [GEARS]). Generalized linear models were used for statistical modeling. According to their initial performance, each participant was categorized as either an innate performer or an underperformer, depending on whether their error tally was above or below the median.

Results and limitations: In round 2, the intervention group had a larger decrease in error rate than the control group, with a risk ratio (RR) of 1.51 (95% confidence interval [CI] 1.07-2.14; p = 0.02). The intervention group also had a greater increase in GEARS score in comparison to the control group, with a mean group difference of 2.15 (95% CI 0.81-3.49; p < 0.01). The interaction effect between innate performers versus underperformers and the intervention was statistically significant for the error rates, at F(1,38) = 5.16 (p = 0.03). Specifically, the intervention had a statistically significant effect on the error rate for underperformers (RR 2.23, 95% CI 1.37-3.62; p < 0.01) but not for innate performers (RR 1.03, 95% CI 0.63-1.68; p = 0.91).

Conclusions: Real-time feedback improved performance globally compared to the control. The benefit of real-time feedback was stronger for underperformers than for trainees with innate skill.

Patient summary: We found that real-time feedback during a training task using a surgical robot improved the performance of trainees when the task was repeated. This feedback approach could help in training doctors in robotic surgery.

Keywords: Feedback; Learning; Mentoring; Robotic surgery; Surgical education.

Fig. 1

Example of the feedback delivery process.

Fig. 2

(A) Model-estimated mean error rate and technical skill score (GEARS) for rounds 1 and 2 for the feedback and control groups. (B) Model-estimated mean error rates for underperformers and innate performers. (C) Model estimated mean technical skill score (GEARS) for underperformers and innate performers. For all plots, error bars represent the 95% confidence interval for the estimates. GEARS = Global Evaluative Assessment of Robotic Surgery.

Fig. 3

Empirical correlation of the System Usability Scale (SUS) score with the error count and technical skills score (GEARS) for underperformers and innate performers. The difference in slope shows that the feedback was more important in helping underperformers than innate performers. GEARS = Global Evaluative Assessment of Robotic Surgery.