Assessment of Surgery Residents' Interpersonal Communication Skills: Validation Evidence for the Communication Assessment Tool in a Simulation Environment

Abstract

Objectives: Although development of trainees' competency in interpersonal communication is essential to high-quality patient-centered surgical care, nontechnical skills present assessment challenges for residency program directors. The Communication Assessment Tool (CAT) demonstrated internal reliability and content validity for general surgery residents, though the tool has not yet been applied in simulation. The study provides validation evidence for using the CAT to assess surgical residents' interpersonal communication skills in simulation scenarios.

Design: Simulations of delivering bad news were completed by 21 general surgery residents during a mandatory communication curriculum. Upon completion of the 10-minute scenario, standardized participants (SPs) assessed performance using the 14-item CAT rating scale and individually provided feedback to residents. Discrete communication behaviors were recorded on video review by a trained blinded observer. The traits emotional intelligence questionnaire short form (TEIQue-SF) was completed by the residents 6 months later. SP-CAT ratings are evaluated with respect to learner characteristics, observed behaviors, and TEIQue results.

Setting: Surgical simulation center in a 900-bed tertiary care hospital.

Participants: General surgery residents were targeted learners. Trauma survivors network volunteers served as SPs, acting as a family member of a patient who developed an intracerebral hemorrhage following a small bowel procedure.

Results: Discrete communication behaviors were reliably assessed by the observer (interrater reliability with trainer: 89% agreement, κ = 0.77). SP-CAT ratings ranged from 34 to 61. Higher SP-CAT ratings were correlated with positive communication behaviors (Spearman ρ = 0.42, p = 0.056). Total TEIQue was positively related to SP-CAT ratings (ρ = 0.42, p = 0.061). The TEIQue emotionality factor was strongly correlated with SP-CAT ratings (ρ = 0.52, p = 0.016).

Conclusions: The CAT demonstrates content validity in a simulation environment with former patients acting as SPs. This study provides validation evidence relating the SP-CAT to discrete observations of communication behaviors by a trained, reliable observer as well as residents' self-reported emotional intelligence traits.

Keywords: Interpersonal and Communication Skills; Patient Care; Professionalism; general surgery; interpersonal and communication skills; residency; simulation training.