The SIMULATE ureteroscopy training curriculum: educational value and transfer of skills

Abstract

Objective: Different simulation modalities may be utilised in a curricular fashion to benefit from the strengths of each training model. The aim of this study is to evaluate a novel multi-modality ureterorenoscopy (URS) simulation curriculum in terms of educational value, content validity, transfer of skills and inter-rater reliability.

Methods: This international prospective study recruited urology residents (n = 46) with ≤ 10 URS experience and no prior simulation training. Participants were guided through each phase of the expert-developed SIMULATE URS curriculum by trainers and followed-up in the operating room (OR). Video recordings were obtained during training. A post-training evaluation survey was distributed to evaluate content validity and educational value, using descriptive statistics. Performance was evaluated using the objective structured assessment of technical skills (OSATS) scale to measure improvement in scores throughout the curriculum. Pearson's correlation coefficient and Cohen's kappa tests were utilised to investigate correlation and agreement between raters.

Results: Participants reported gaining OR-transferrable skills (Mean: 4.33 ± 0.67) and demonstrated marked improvement in throughout the curriculum, transferred to the OR for both semi-rigid URS (p = 0.004) and flexible URS (p = 0.007). 70% of participants were successfully followed-up in the OR (n = 32). No differences were identified with the additional use of fresh frozen cadavers (p = 0.85, p = 0.90) and the URO Mentor VR simulator (p = 0.13, p = 0.22). A moderate level of correlation was noted on the video OSATS assessments, between two expert assessors (r = 0.70), but a poor agreement with the live rating.

Conclusion: The SIMULATE URS training curriculum received high educational value from participants, who demonstrated statistically significant improvement with consecutive cases throughout the curriculum and transferability of skills to the OR in both semi-rigid and flexible URS.

Keywords: Curriculum; Education; Simulation; Ureterorenoscopy; Urology training.

Fig. 1

An overview of the study methodology, the utilised training curriculum and participant timeline. Five training sessions were conducted with novices in ureteroscopy (n = 46). Performance evaluation took place live and through video assessments. Participants were followed-up in their home institutions for transfer of skills. OR operating room, OSATS objective structured assessment of technical skills, URS ureteroscopy, fURS flexible ureterorenoscopy

Fig. 2

Improvement in mean OSATS (%) scores over consecutive scenarios and operating room (OR). B Bench, FFC fresh frozen cadavers, OR operating room, VR virtual reality

Fig. 3

Inter-rater reliability of all ureteroscopy cases between the video assessment of two expert raters (a, b) and live rating. A Pearson’s coefficient and Cohen’s kappa was performed for all cases to assess correlation and agreement, respectively. B bench, FFC fresh frozen cadavers, OR operating room, VR virtual reality