Supervisors' self-assessment of feedback skills: a psychometric validation study of the English version of the SwissSETQ questionnaire for supervisors

Abstract

Study aims: We created an instrument to assess the supervisors' perspective on their feedback behaviour to residents and investigated its validity. Our instrument is based on the SETQsmart, a Dutch instrument for assessing the quality of supervision in clinical training and the SwissSETQ, its German adaptation for residents. Our instrument is in English to ensure relevance across all Swiss language regions. The study specifically sought: to replicate the factor structure of the original trainee questionnaire for supervisors; to verify the alignment of SwissSETQ and SETQsmart domains with the factor structure; and to evaluate the psychometric properties of the English version.

Methods: The original SwissSETQ was translated into English, maintaining the Swiss context and local language usage. The questionnaire was adjusted to reflect the supervisor's perspective. The translated questionnaire was then distributed among supervisors in all Swiss cardiology training sites, and data were collected using the SoSci Survey platform between March and April 2024. The statistical analysis, including exploratory factor analysis (EFA) with promax rotation, Bartlett's test of sphericity, Kaiser-Meyer-Olkin (KMO) coefficient and psychometric evaluation, was conducted using R software.

Results: Of approximately 600 cardiology supervisors in Switzerland, 207 responded, with 135 valid cases remaining after data cleaning. The factor analysis identified three factors: Teaching structure, Attitude of the supervisor and Role modelling, resulting in a shortened 23-item questionnaire. The Kaiser-Meyer-Olkin coefficient was 0.83, and Bartlett's test was significant, confirming data suitability for factor analysis. The factors demonstrated high internal consistency, with Cronbach's α values of 0.89, 0.77 and 0.87, respectively. The partial credit model indicated the need for a revised 5-point Likert scale for better response distribution. No significant differences were found between factors and sociodemographic variables, suggesting the English version's applicability across all Swiss language regions.

Conclusions: The study investigated the English-translated and supervisor-adapted version of the SwissSETQ, demonstrating good psychometric properties and a clear factor structure. The instrument is suitable for use across different Swiss language regions, enhancing its utility in a multilingual context. The findings support the potential of the SwissSETQ to facilitate cross-cultural and cross-linguistic collaboration in medical training. Future research should explore additional factors influencing teaching quality, such as work environment and supervisor motivation.