What's the use of faculty development? Program evaluation using retrospective self-assessments and independent performance ratings

Abstract

Background: The assessment of the effectiveness of faculty development programs is increasingly important in medical schools and academic medical centers but is difficult to accomplish.

Purpose: We investigated the usefulness of retrospective self-assessments by program participants in combination with independent ratings of teaching performance by their trainees.

Methods: We used a single sample, prepost intervention design using multiple measures. Our assessment instruments were based on our institution's accepted teaching competencies. We measured participants' self-assessments of their teaching competencies before the program and their retrospective self-assessed improvements in these competencies after the program. We also used independent ratings of the participants' teaching competencies before and after their involvement in the program, as rated by their own trainees (fellows, residents, and medical students). Selected teaching competencies comprised the intended learning outcomes of the faculty development program.

Results: Participants' preprogram self-assessments showed that the program was appropriately matched to several topics identified as needy, but also included topics that participants did not identify as needs. The retrospective self-assessments showed improvements in teaching skills that previously were identified as needs, as well as those in which participants originally felt quite competent. The independent ratings by trainees showed overall positive improvements (some significantly). The retrospective self-assessed improvements correlated positively with the independent ratings by their trainees (p < .01).

Conclusions: This evaluation strategy showed that the faculty development program improved the teaching competencies of the participants. Both the program participants' retrospective self-assessments and the independent ratings by their trainees showed postprogram improvements and were positively intercorrelated. The use of these multiple measures is a viable approach to evaluate the impact of a faculty development program. Potentially either approach could be used, but in combination, they provide a feasible, valid, and reliable evaluation.