Using videotaped benchmarks to improve the self-assessment ability of family practice residents

Abstract

Purpose: To address methodologic and statistical problems of previous studies of self-assessment by exposing participants to relevant standards, anchoring rating scales, and providing practice in the use of the assessment tool.

Method: Fifty first- and second-year family practice residents performed a ten-minute patient interview with a difficult communication problem. Following each interview, the resident and two experts independently evaluated the resident's communication skills. The resident was then shown a videotape of four performances (ranging in quality from poor to good) of the same scenario. The resident evaluated the communication skills displayed in each performance and then reevaluated his or her own performance.

Results: The correlation between experts' evaluations and residents' self-evaluations was moderate immediately after the interview (r = 0.38) but increased significantly after the residents viewed the videotape (r = 0.52). This effect was more pronounced for first-year residents (0.22 to 0.45) than for second-year residents (0.53 to 0.65), although the difference was not significant. Post-hoc analysis revealed that neither initial nor post-benchmark self-assessment ability was related to the ability to accurately evaluate the benchmarks in a manner consistent with the experts.

Conclusions: The ability to self-assess does not seem strongly tied to the ability to assess the performances of others on the same task. Nonetheless, providing a set of benchmarks against which trainees can compare their own performances improves their ability to self-evaluate even if the qualities of the benchmarks are not explicitly identified.