Training for Failure: A Simulation Program for Emergency Medicine Residents to Improve Communication Skills in Service Recovery

Abstract

Objectives: Service failures such as long waits, testing delays, and medical errors are daily occurrences in every emergency department (ED). Service recovery refers to the immediate response of an organization or individual to resolve these failures. Effective service recovery can improve the experience of both the patient and the physician. This study investigated a simulation-based program to improve service recovery skills in postgraduate year 1 emergency medicine (PGY-1 EM) residents.

Methods: Eighteen PGY-1 EM residents participated in six cases that simulated common ED service failures. The patient instructors (PIs) participating in each case and two independent emergency medicine (EM) faculty observers used the modified Master Interview Rating Scale to assess the communication skills of each resident in three simulation cases before and three simulation cases after a service recovery debriefing. For each resident, the mean scores of the first three cases and those of the last three cases were termed pre- and postintervention scores, respectively. The means and standard deviations of the pre- and postintervention scores were calculated by the type of rater and compared using paired t-tests. Additionally, the mean scores of each case were summarized. In the framework of the linear mixed-effects model, the variance in scores from the PIs and faculty observers was decomposed into variance contributed by PIs/cases, the program effect on individual residents, and the unexplained variance. In reliability analyses, the intraclass correlation coefficient between rater types and the 95% confidence interval were reported before and after the intervention.

Results: When rated by the PIs, the pre- and postintervention scores showed no difference (p = 0.852). In contrast, when scored by the faculty observers, the postintervention score was significantly improved compared to the preintervention score (p < 0.001). In addition, for the faculty observers, the program effect was a significant contributor to the variation in scores. Low intraclass correlation was observed between rater groups.

Conclusions: This innovative simulation-based program was effective at teaching service recovery communication skills to residents as evaluated by EM faculty, but not PIs. This study supports further exploration into programs to teach and evaluate service recovery communication skills in EM residents.

Figure 1

Service recovery framework.

Figure 2

Scatter plots of mean scores pre‐ or postintervention between rater types and ICCs along with 95% CIs. ICC = intraclass correlation coefficient.