Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2008 May;90(4):286-90.
doi: 10.1308/003588408X286008.

Self-assessment of technical skill in surgery: the need for expert feedback

V A Pandey  1 , J Hn Wolfe, S A Black, M Cairols, C D Liapis, D Bergqvist; European Board of Vascular Surgery
Affiliations

Self-assessment of technical skill in surgery: the need for expert feedback

V A Pandey et al. Ann R Coll Surg Engl. 2008 May.

Abstract

Introduction: Technical skill has been formally assessed in the Fellow of the European Board of Vascular Surgery Examinations (FEBVS) since 2002. The aim of this study was to examine the relationship between expert assessment and trainee self-assessment.

Materials and methods: Forty-two examination candidates performed a saphenofemoral junction (SFJ) ligation and an anterior tibial anastomosis on a synthetic simulation. Each candidate was rated by two examiners using a validated rating scale for their generic surgical skill for both procedures. Candidates then anonymously rated their own performance using the same scale. Parametric tests were used in the statistical analysis; a P-value < 0.05 was considered significant.

Results: The maximum mark in each assessment was 40; 24 was considered a competent score. The interobserver correlation for examiners marks were high (SFJ ligation, alpha = 0.68; distal anastomosis, alpha = 0.76). Examiners' marks were averaged. The mean examiner score for the SFJ ligation station was 27.8 (SD = 4.1) with 36 candidates (85.8%) attaining a competent score. The mean self-assessment score for this station was 30.7 (SD = 4.66). The mean examiners' marks for the distal anastomosis station was 29.2 (SD = 4.2); 39 candidates (92.8%) attained a competent score. The mean self-assessment score was 32.1 (SD = 4.0). There was no correlation between examiner and self-assessment scores in either station (Pearson's correlation coefficient: SFJ, r = 0.045, P = NS); distal anastomosis, r = 0.089, P = NS). Bland and Altman plots assessed the agreement between examiner and self-assessment. These showed candidates marked themselves higher than examiners with a mean difference of 2.9 marks in each station.

Conclusions: Candidates' self-assessment and expert independent assessment correlate poorly. Trainees overestimate their ability according to independent assessment; regular technical feedback during training is, therefore, essential.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Bespoke model of the human saphenofemoral junction. Inset: the saphenofemoral junction dissected and the tributaries ligated.
Figure 2
Figure 2
Synthetic leg model. The anterior tibial artery is exposed for a distal arterial anastomosis.
Figure 3
Figure 3
Saphenofemoral junction ligation. Correlation between examiner (horizontal axis) and self-assessment (vertical axis). r = 0.045; P = NS.
Figure 4
Figure 4
Distal anastomosis. Correlation between examiner (horizontal axis) and self-assessment (vertical axis). r = 0.089; P = NS.
Figure 5
Figure 5
Bland and Altman plot for saphenofemoral junction ligation. The average of the examiner and self-assessment scores on the horizontal axis and the difference between examiner and self-assessment scores on the vertical axis. The central horizontal line represents the mean difference between the two scores (2.9 marks).
Figure 6
Figure 6
Bland and Altman plot for distal anastomosis. The average of the examiner and self-assessment scores on the horizontal axis and the difference between examiner and self-assessment scores on the vertical axis. The central horizontal line represents the mean difference between the two scores (again the difference was 2.9 marks).
See this image and copyright information in PMC

Comment in

  • Simulators in self-assessment.
    Smith JK, Baxendale B, Ferguson E, Maxwell-Armstrong CA. Smith JK, et al. Ann R Coll Surg Engl. 2009 Jan;91(1):90. doi: 10.1308/003588409X359187. Ann R Coll Surg Engl. 2009. PMID: 19126343 Free PMC article. No abstract available.

References

    1. Harrington JP, Murnaghan JJ, Regehr G. Applying a relative ranking model to the self-assessment of extended performances. Adv Health Sci Educ Theory Pract. 1997;2:17–25. - PubMed
    1. Moorthy K, Munz Y, Adams S, Pandey V, Darzi A. Self-assessment of performance among surgical trainees during simulated procedures in a simulated operating theater. Am J Surg. 2006;192:114–8. - PubMed
    1. Ward M, MacRae H, Schlachta C, Mamazza J, Poulin E, Reznick R, et al. Resident self-assessment of operative performance. Am J Surg. 2003;185:521–4. - PubMed
    1. Falchikov N, Boud D. Student self-assessment in higher education: a meta-analysis. Rev Educ Res. 1989;59:395–430.
    1. Risucci DA, Tortolani AJ, Ward RJ. Ratings of surgical residents by self, supervisors and peers. Surg Gynecol Obstet. 1989;169:519–26. - PubMed