Learning while evaluating: the use of an electronic evaluation portfolio in a geriatric medicine clerkship

Abstract

Background: Electronic evaluation portfolios may play a role in learning and evaluation in clinical settings and may complement other traditional evaluation methods (bedside evaluations, written exams and tutor-led evaluations).

Methods: 133 third-year medical students used the McGill Electronic Evaluation Portfolio (MEEP) during their one-month clerkship rotation in Geriatric Medicine between September 2002 and September 2003. Students were divided into two groups, one who received an introductory hands-on session about the electronic evaluation portfolio and one who did not. Students' marks in their portfolios were compared between both groups. Additionally, students self-evaluated their performance and received feedback using the electronic portfolio during their mandatory clerkship rotation. Students were surveyed immediately after the rotation and at the end of the clerkship year. Tutors' opinions about this method were surveyed once. Finally, the number of evaluations/month was quantified. In all surveys, Likert scales were used and were analyzed using Chi-square tests and t-tests to assess significant differences in the responses from surveyed subjects.

Results: The introductory session had a significant effect on students' portfolio marks as well as on their comfort using the system. Both tutors and students reported positive notions about the method. Remarkably, an average (+/- SD) of 520 (+/- 70) evaluations/month was recorded with 30 (+/- 5) evaluations per student/month.

Conclusion: The MEEP showed a significant and positive effect on both students' self-evaluations and tutors' evaluations involving an important amount of self-reflection and feedback which may complement the more traditional evaluation methods.

Figure 1

A and B. Effect of the introductory session on students' performance in their MEEP and POM evaluations. The figure shows the average scores of MEEP (panel A) and POM form (panel B) for the 77 students who received an introductory session vs. 47 who did not receive it. A significant difference was found between both groups in the MEEP while no difference for POM marks was found (*p < 0.001). C. Student ratings of the electronic portfolio as an evaluation tool. Using a 5-point Likert scale students were asked to rate the MEEP as an evaluation tool using a scale ranging from Poor to Excellent both immediately and post-clerkship. The figure shows a significant positive change in the number of students who considered the MEEP as very good or good evaluation tool immediately after their rotation versus at the post-clerkship survey (p < 0.04).

Figure 2

Number of evaluating postings per month by students and tutors. Throughout the academic year 2003–2004 we had an average of 520 ± 70 postings/month with a tutor/student ratio of 1.3 indicating a slightly higher proportion of students' postings.