Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program

Abstract

Introduction: The National Training Program for laparoscopic colorectal surgery (LCS) provides supervised training to colorectal surgeons in England. The purpose of this study was to create, validate, and implement a method for monitoring training progression in laparoscopic colorectal surgery that met the requirements of a good assessment tool.

Methods: A generic scale for different tasks in LCS was created under the guidance of a national expert group. The scores were defined by the extent to which the trainees were dependent on support (1 = unable to perform, 5 = unaided (benchmark), 6 = proficient). Trainers were asked to rate their trainees after each supervised case; trainees completed a similar self-assessment form. Construct validity was evaluated comparing scores of trainees at different experience levels (1-5, 6-10, 11-15, 16+) using the Wilcoxon signed-rank test and ANOVA. Internal consistency was determined by Crohnbach's alpha, interrater reliability by comparing peer- and self-assessment (interclass correlation coefficient, ICC). Proficiency gain curves were plotted using CUSUM charts.

Results: Analysis included 610 assessments (333 by trainers and 277 by trainees). There was high interrater reliability (ICC = 0.867), internal consistency (α = 0.920), and construct validity [F(3,40) = 6.128, p < 0.001]. Detailed analysis of proficiency gain curves demonstrates that theater setup, exposure, and anastomosis were performed independently after 5 to 15 sessions, and the dissection of the vascular pedicle took 24 cases. Mobilization of the colon and of the splenic/hepatic flexure took more than 25 procedures. Median assessment time was 3.3 (interquartile range (IQR) 1-5) minutes and the tool was accepted as useful [median score 5 of 6 (IQR 4-5)].

Discussion: A valid and reliable monitoring tool for surgical training has been implemented successfully into the National Training Program. It provides a description of an individualized proficiency gain curve in terms of both the level of support required and the competency level achieved.