Acceptance and validity of the methods used to implement a competency based medical education programme in an Intensive Care Department of a teaching referral center

Abstract

Objectives: 1. To determine the satisfaction of tutors and residents with a specific methodology used to implement CoBaTrICE. 2. To determine the reliability and validity of the global rating scales designed ad hoc to assess the performance of the residents for training purposes.

Design: Prospective cohort study.

Participants: All the residents and tutors of the ICU Department of the Hospital Universitario y Politécnico la Fe de Valencia.

Intervention: CoBaTrICE implementation started in March 2016, it was based on: (1) Training the tutors in feedback techniques; (2) Performing multiple objective and structured work based assessments to achieve the competences of the program; and (3) The use of an electronic portfolio to promote learning reflection and to collect the evidence that learning was taking place.

Methods: The acceptance of CoBaTrICE was explored through a satisfaction survey conducted after 9 months of implementation of the training program. The 15 residents and 5 tutors of the ICU Department were asked about the methodology of the formative assessments, the quality of the feedback, self-learning regulation and the electronic portfolio usefulness. The validity of the global rating scales was assessed through the tests alfa de Cronbach, reliability and generalizability indexes, and intraclass correlation coefficient.

Results: The implementation of CoBaTrICE was satisfactory in all the dimensions studied. The global rating scales used for formative purposes showed reliability and validity.

Conclusions: The methodology used to implement CoBaTrICE was highly valued by tutors and residents. The global rating scales used for formative purposes showed reliability and validity.

Keywords: CoBaTrICE; Competency based medical education; Electronic portfolio; Escalas de valoración; Evaluación formativa; Formación médica basada en competencias; Formative assessment; Global rating scales; Intensive Care Medicine; Medicina Intensiva; Portafolio electrónico; Reliability and validity; Validez y fiabilidad.