Quantitating qualitative issues in residency training: development and testing of a scaled program evaluation questionnaire

Abstract

Objectives: To develop and test a scaled program evaluation questionnaire focusing on resident satisfaction with workload, learning environment, and stress.

Design/participants: Phase 1: A cross-section of 92 residents from five programs completed questionnaires for factor analysis and descriptive statistics. Phase 2: A three-year prospective study of questionnaire responses in a single program. QUESTIONNAIRE DEVELOPMENT: After extensive literature review, 33 Likert-scaled statements were written, dealing with situational, personal, and professional issues. After pilot testing, the 92 questionnaires obtained in Phase 1 were factor analyzed, resulting in three distinct scales: workload, faculty/learning environment, and stress.

Interventions: Program changes in Phase 2 included the introduction of a night float between 1989 and 1990, and an increase in senior resident call between 1990 and 1991.

Results: Phase 1: The first-year residents reported significantly less stress and generally greater satisfaction with workload and learning environment than did the second- or third-year residents. Marked differences between programs were also present in the mean scores on all three scales. Phase 2: The introduction of a night float did not significantly affect response to the questionnaire, but the increase in nights on call significantly increased stress and dissatisfaction with the faculty/learning environment.

Conclusions: This study demonstrates the techniques needed to develop and use scaled program evaluation questionnaires. It is hoped that widespread use and validation of such instruments may result in greater responsiveness to the needs of trainees and more facilitative environments in which to acquire medical knowledge.