Randomized Trial of Smartphone-Based Evaluation for an Obstetrics and Gynecology Clerkship

Abstract

Objective: We hypothesized that compared to paper evaluations, a smartphone-based quick response (QR) evaluation tool would improve timeliness of feedback, enhance efficacy of giving and receiving feedback, and be as easy to use.

Design: We performed a randomized controlled trial of student and instructor experience with two evaluation tools in the OB/GYN clerkship at University of Washington School of Medicine (UWSOM). Sites were randomized to the QR or paper tool; students at QR sites received individualized QR codes at the beginning of the clerkship. Instructors and students completed postintervention surveys regarding the evaluation tool and associated feedback. We compared responses between groups using chi-squared tests.

Setting: Participating clerkship sites included primary, tertiary, private practice and institutional settings affiliated with the University of Washington in the Washington, Wyoming, Alaska, Montana and Idaho region.

Participants: Of the 29 OB/GYN UWSOM clerkship sites, 18 agreed to participate and were randomized. Of 29 eligible instructors, 25 (86%) completed the survey, with n = 18 using QR and n = 7 using paper. Of 161 eligible students, 102 (63%) completed the survey, with n = 54 using QR and n = 48 using paper.

Results: Compared to those using paper evaluations, instructors using QR evaluations were significantly more likely to agree that the evaluation tool was easy to understand (100% QR vs 43% paper, p = 0.002), the tool was effective in providing feedback (78% QR vs 29% paper, p = 0.002), and they felt comfortable approaching students with the tool (89% QR vs 43% paper, p = 0.002). Compared to those using paper evaluations, students using QR evaluations were less likely to agree the tool was effective in eliciting feedback (QR 43% vs paper 55%, p = 0.042).

Conclusion: Instructors found QR evaluations superior to paper evaluations for providing feedback to medical students, whereas students found QR evaluations less effective for feedback.

Keywords: Interpersonal and Communication Skills; Medical Knowledge; Patient Care; Practice-Based Learning and Improvement; evaluation; feedback; medical education; smartphone technology.