Multiple Objective Measures of Skill (MOMS): a new approach to the assessment of technical ability in surgical trainees

Abstract

Objective: The assessment of surgical technical skills has become an important topic in recent years. This study presents the validation of a 6-task skills examination for junior surgical trainees (at the level of the Membership of the Royal College of Surgeons).

Summary background data: Six tasks were evaluated in a project that also examined the feasibility of this method of assessment. The tasks were knowledge of sutures and instruments; knowledge of surgical devices; knot formation; skin-pad suturing, closure of an enterotomy; excision of a skin lesion; and laparoscopic manipulation. Comparisons were made between a group of junior trainees (n = 13), and a group of seniors (n = 8).

Results: Each of the 6 tasks was able to be used to discriminate between the 2 groups. In all, there were 19 primary analyses across the 6 tasks, and 17 of these showed significant differences between the groups (P values ranging from 0.037 to < 0.001). There was generally a strong correlation between the analyses, and when a mean rank was calculated, the difference between groups was significant (P = 0.005 on Mann-Whitney U test; mean ranks 13.9 and 6.3 [of 21], for juniors and seniors respectively). Reliability of the 6-task assessment was very good at 0.70 (Cronbach's Alpha).

Conclusions: A skills examination is a feasible and effective method of assessing the technical ability of basic surgical trainees.

FIGURE 1. Total score for task 1 by group. Maximum possible score was 25. P = 0.024 (MWU).

FIGURE 2. Movement data for the 2/O Polysorb-at-depth component of task 2 (knot formation). P = 0.001 (MWU). The position of a single outlier in the junior group (with a score of 200) is not shown in this graph.

FIGURE 3. Movement data for the skin-pad suture task (task 3). The data for simple and mattress sutures are presented for each group. Differences between groups were significant (P = 0.001 for simple suture, and P < 0.001 for mattress suture; MWU). The single outlier for the simple suture task in the senior group is indicated.

FIGURE 4. Checklist score results for task 4 (enterotomy closure). P = 0.19 (MWU).

FIGURE 6. Mean rank across 18 analyses, between the 2 groups. P = 0.005 (MWU).

FIGURE 5. Trial-by-trial results for the MIST VR laparoscopic acquire place component of task 6.