Objective structured assessment of technical skills in elliptical excision repair of senior dermatology residents: a multirater, blinded study of operating room video recordings

Abstract

Importance: Surgical training in dermatology residencies has increased, and there is growing interest in measuring resident competence more precisely. This study applies the use of the objective structured assessment of technical skills (OSATS) to measure competence in dermatologic surgery.

Objective: To evaluate the utility of OSATS as a tool for measuring surgery skills during dermatology residency training.

Design, setting, and participants: Multirater, blinded review and ratings of taped video recordings of elliptical excisions performed by senior dermatology residents applying for procedural or Mohs surgery fellowships.

Main outcomes and measures: Ratings on a specific OSATS measure, including global rating scale and task checklist.

Results: Twelve videos, representing approximately 20% of fellowship applicants during 2009-2010, were rated. Raters agreed on 272 of 288 subscore ratings (94.4%). Mean global ratings were 4 or 5 for all categories, except time and motion, which had a rating of 3. Task checklist ratings had a mean of 14.5 and a mode of 16 (perfect score), with eversion least often performed successfully. No association was found between a resident's scores and the number of surgical rotation months or between scores and the number of Mohs surgeons at the home institution.

Conclusions and relevance: Senior dermatology residents preparing for surgery fellowships are highly skilled in performing elliptical excisions and bilayered repairs. The OSATS appears useful and reliable for the evaluation of dermatologic surgery skills.