A Mixed-methods Comparison of Participant and Observer Learner Roles in Simulation Education

Abstract

Background: Traditional simulation-based education prioritizes participation in simulated scenarios. The educational impact of observation in simulation-based education compared with participation remains uncertain. Our objective was to compare the performances of observers and participants in a standardized simulation scenario.

Methods: We assessed learning differences between simulation-based scenario participation and observation using a convergent, parallel, quasi-experimental, mixed-methods study of 15 participants and 15 observers (N = 30). Fifteen first-year residents from six medical specialties were evaluated during a simulated scenario (cardiac arrest due to critical hyperkalemia). Evaluation included predefined critical actions and performance assessments. In the first exposure to the simulation scenario, participants and observers underwent a shared postevent debriefing with predetermined learning objectives. Three months later, a follow-up assessment using the same case scenario evaluated all 30 learners as participants. Wilcoxon signed rank and Wilcoxon rank sum tests were used to compare participants and observers at 3-month follow-up. In addition, we used case study methodology to explore the nature of learning for participants and observers. Data were triangulated using direct observations, reflective field notes, and a focus group.

Results: Quantitative data analysis comparing the learners' first and second exposure to the investigation scenario demonstrated participants' time to calcium administration as the only statistically significant difference between participant and observer roles (316 seconds vs. 200 seconds, p = 0.0004). Qualitative analysis revealed that both participation and observation improved learning, debriefing was an important component to learning, and debriefing closed the learning gap between observers and participants.

Conclusions: Participants and observers had similar performances in simulation-based learning in an isolated scenario of cardiac arrest due to hyperkalemia. Findings support current limited literature that observation should not be underestimated as an important opportunity to enhance simulation-based education. When paired with postevent debriefing, scenario observers and participants may reap similar educational benefits.

Figure 1

4Cs quantitative study design flow diagram. 4Cs = Common Critical Care Curriculum.

Figure 2

Learner performance assessment tool.

Figure 3

Graphical depiction of two rater performance ratings for each clinical domain.