Using the Objective Structured Assessment of Technical Skills (OSATS) global rating scale to evaluate the skills of surgical trainees in the operating room

Abstract

Purpose: The education of surgical trainees should be based on an accurate evaluation of their surgical skill levels. In our hospital, the Objective Structured Assessment of Technical Skills (OSATS) is used for this purpose. We conducted this study to demonstrate the validity and accuracy of the OSATS for assessing surgical skills in the operating room (OR) setting.

Methods: Between January, 2007 and December, 2010, the OSATS global rating scale was used to assess several operations in which surgical trainees participated. We assessed ten surgical trainees who participated as the main surgeon or first assistant, and studied the correlation between their postgraduate year and their OSATS score.

Results: The median score of the global rating scale for each trainee improved with each year of experience. The median scores of all trainees in postgraduate years 3, 4, and 5 were significantly different (p < 0.001 for both the main surgeon and first assistant roles; Kruskal-Wallis test).

Conclusion: Using the OSATS global rating scale to assess the surgical skills of trainees in the OR was feasible and effective.

Fig. 1

The global rating scale used in the Objective Structured Assessment of Technical Skills (OSATS) [2], which we used to score the skills of each surgical trainee in performing or assisting in real operations. Full marks are 35 points on 7 items and 30 points on 6 items, respectively, as a surgeon and as an assistant (in the case of assistant, ‘Use of Assistant’ is excluded from the scoring)

Fig. 2

Changes in the global rating scale for each trainee as a surgeon. Above low difficulty, below intermediate difficulty

Fig. 3

Changes in the global rating scale for each trainee as a first assistant. Above low difficulty, middle intermediate difficulty, below high difficulty

Fig. 4

Correlation between the postgraduate year and the score. The scores were compared among the second terms of postgraduate years 3, 4, and 5. Above as a surgeon. Below as a first assistant. The differences in the scores of these three groups were significant (Kruskal–Wallis test, <0.001 and <0.001, respectively)