Novel Mobile App Allows for Fast and Validated Intraoperative Assessment of Otolaryngology Residents

Abstract

Evaluation of resident operative skills is challenging in the fast-paced operating room environment and limited by lack of validated assessment metrics. We describe a smartphone-based app that enables rapid assessment of operative skills. Accreditation Council for Graduate Medical Education (ACGME) otolaryngology taxonomy surgical procedures (n = 593) were uploaded to the software platform. The app was piloted over 1 month. Outcomes included (1) completion of evaluation, (2) time spent completing the evaluation, and (3) quantification of case complexity, operative autonomy, and performance. During the study, 12 of 12 procedures, corresponding to 3 paired evaluated by the resident/attending dyad. Mean ± SD time of evaluation completion was 98.0 ± 24.2 and 123.0 ± 14.0 seconds for the resident and attending, respectively. Mean time between resident and attending evaluation completion was 27.9 ± 26.8 seconds. Resident and attending scores for case complexity, operative autonomy, and performance were strongly correlated (P < .0001). Rapid evaluation of resident intraoperative performance is feasible using smartphone-based technology.

Keywords: education; mobile; otolaryngology residency; surgical education.

Figure 1.

Demonstration of System for Improving and Measuring Procedural Learning (SIMPL) app evaluation. Top right describes the validated “Zwisch” scale. Bottom left and middle screens ask questions related to case complexity and preparation.

Figure 2.

Time to complete evaluations. Time to complete evaluation on System for Improving and Measuring Procedural Learning (SIMPL) app (n = 12 procedures, and 3 paired evaluations).