A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance
- PMID: 18063265
- DOI: 10.1016/j.jsurg.2007.06.015
A new web-based operative skills assessment tool effectively tracks progression in surgical resident performance
Abstract
Purpose: The study aim was to demonstrate that a new database tool for assessment of surgical resident operative skills discerns predictable progression in those skills over successive residency years for specific index case types.
Methods: A Web-based interactive database (OpRate) was used to assess selected aspects of resident operative performance as determined by supervising attending surgeons in a medium-sized residency (5-6 residents per postgraduate year [PGY]). This assessment consisted of (1) 3 questions pertaining to patient information, technical, and disease-specific preparedness; (2) 4 laparoscopic technical skills questions pertaining to tissue handling, dexterity, planning, and ability to function independently; and (3) similar open technical skills questions, with the addition of 2 questions defining knot tying ability. Two years of assessment data were examined for cholecystectomy (CH), appendectomy (AP), colon resection (CR), ventral hernia repair (VH), and inguinal hernia repair (IH). Mean scores for total, technical, and preparedness responses, as well as each response area were compared for successive training years for each case type. Mean performance data between postgraduate years were compared by ANOVA, and interitem reliability was assessed by Cronbach's alpha determinations.
Results: OpRate data for 579 cases (142 CH, 67 AP, 73 CR, 202 IH, and 95 VH) were examined. Significant incremental increases in open and laparoscopic technical skills scores by training year were observed for all case types (ANOVA, p < 0.0001). Individual technical skills as well as technical and disease-specific preparedness response areas also demonstrated significant improvement by successive training year. Cronbach's alpha determinations were 0.80-0.94 for the preparedness test items and the skills performance scores for all assessed procedures.
Conclusions: Our early results show that the OpRate assessment tool is effective in identifying expected changes in operative performance across successive training years, with a satisfactory level of internal consistency for the test items. As such, the use of this database tool may offer the opportunity to (1) define performance benchmarks for specific levels of training and (2) identify areas where focused training may be required for specific residents.
Similar articles
-
Boot cAMP: educational outcomes after 4 successive years of preparatory simulation-based training at onset of internship.J Surg Educ. 2012 Mar-Apr;69(2):242-8. doi: 10.1016/j.jsurg.2011.08.007. J Surg Educ. 2012. PMID: 22365874
-
Feasibility, reliability and validity of an operative performance rating system for evaluating surgery residents.Surgery. 2005 Oct;138(4):640-7; discussion 647-9. doi: 10.1016/j.surg.2005.07.017. Surgery. 2005. PMID: 16269292
-
Evaluation of resident laparoscopic performance using global operative assessment of laparoscopic skills.J Am Coll Surg. 2007 Feb;204(2):308-13. doi: 10.1016/j.jamcollsurg.2006.11.010. Epub 2006 Dec 27. J Am Coll Surg. 2007. PMID: 17254935
-
Assessment of resident operative performance using a real-time mobile Web system: preparing for the milestone age.J Surg Educ. 2014 Nov-Dec;71(6):e41-6. doi: 10.1016/j.jsurg.2014.06.008. Epub 2014 Jul 16. J Surg Educ. 2014. PMID: 25037504 Review.
-
Assessing the skills of surgical residents using simulation.J Surg Educ. 2008 Mar-Apr;65(2):77-83. doi: 10.1016/j.jsurg.2007.11.003. J Surg Educ. 2008. PMID: 18439524 Review.
Cited by
-
Competency-Based Education in Minimally Invasive and Robotic Colorectal Surgery.Clin Colon Rectal Surg. 2021 May;34(3):155-162. doi: 10.1055/s-0040-1718683. Epub 2021 Mar 29. Clin Colon Rectal Surg. 2021. PMID: 33814997 Free PMC article. Review.
-
Development, validation and implementation of a monitoring tool for training in laparoscopic colorectal surgery in the English National Training Program.Surg Endosc. 2011 Apr;25(4):1136-42. doi: 10.1007/s00464-010-1329-y. Epub 2010 Sep 11. Surg Endosc. 2011. PMID: 20835723
-
Tracking Residents' Surgical Outcomes Using Data from the Quality In-Training Initiative.J Surg Educ. 2024 Aug;81(8):1110-1118. doi: 10.1016/j.jsurg.2024.05.012. Epub 2024 Jun 1. J Surg Educ. 2024. PMID: 38825561 Free PMC article.
-
Toward an objective assessment of technical skills: a national survey of surgical program directors in Saudi Arabia.Adv Med Educ Pract. 2012 Oct 10;3:97-104. doi: 10.2147/AMEP.S31720. Print 2012. Adv Med Educ Pract. 2012. PMID: 23762007 Free PMC article.
-
How can surgical skills in laparoscopic colon surgery be objectively assessed?-a scoping review.Surg Endosc. 2022 Mar;36(3):1761-1774. doi: 10.1007/s00464-021-08914-z. Epub 2021 Dec 6. Surg Endosc. 2022. PMID: 34873653 Free PMC article.
MeSH terms
LinkOut - more resources
Full Text Sources
Other Literature Sources
Research Materials