Nontechnical Skills Assessment in Acute Care Trauma Simulations: A Mixed Methods Approach Using Eye Tracking and Behavioral Marker Systems

Abstract

Introduction: The condition of trauma patients and the urgent need for timely resuscitation present unique challenges to trauma teams. These difficulties are exacerbated for military trauma teams in combat environments. Consequently, there is a need for continued improvement of nontechnical skills (NTS) training for trauma teams. However, current approaches to NTS assessment rely on subjective ratings, which can introduce bias. Accordingly, objective methods of NTS evaluation are needed. Eye-tracking (ET) methods have been applied to studying communication, situation awareness, and leadership in many health care settings, and could be applied to studying physicians' NTS during trauma situations. In this study, we aimed to assess the relationship between trauma team leaders' objective gaze patterns and subjective expert NTS ratings during patient care simulations.

Materials and methods: After Institutional Review Board approval, 9 trauma teams from first-year post-graduate general surgery and emergency medicine residents were recruited to participate in 1 of 2 trauma simulations (a difficult airway case and a multi-patient trauma). Each scenario lasted approximately 15 minutes. All team leaders wore a mobile ET system to evaluate gaze metrics-time to first fixation (TTFF), average fixation duration (AFD), and total percentage of the scenario (TPS) focused on Areas of Interest (AOI), which included patient, care team, diagnostic equipment, and patient care equipment. Trained faculty raters completed the Non-Technical Skills for Surgeons (NOTSS) assessment tool and the Trauma Non-Technical Skills (T-NOTECHS) scale. One-way analysis of variance, Kruskal-Wallis, and appropriate post-hoc pairwise comparison tests were run to assess differences between ET metrics across AOI groups. Spearman's Rho tests were used to assess correlations between ET and subjective NTS ratings.

Results: Compared to other NTS domains, trauma teams scored relatively poorly on communication across both T-NOTECHS (3.29$ \pm $0.61, maximum = 5) and NOTSS (2.87$ \pm $0.66, maximum = 4). We found significant differences in trauma team leaders' TTFF between teammates and the patient (Team: 1.56 vs Patient: 29.82 seconds, P < .001). TTFF on the diagnostic equipment was negatively correlated (P < .05) to multiple measures of subjective NTS assessments. There were no significant differences in AFD between AOIs, and AFD on teammates was positively correlated (P < .05) to communication and teamwork. There were significant differences in TPS across most AOI pairs (P < .05), and the average TPS fixated was highest on the patient (32%). Finally, there were several significant correlations between additional ET and NTS metrics.

Conclusions: This study utilized a mixed methods approach to assess trauma team leaders' NTS in simulated acute care trauma simulations. Our results provide several objective insights into trauma team leaders' NTS behaviors during patient care simulations. Such objective insights provide a more comprehensive understanding of NTS behaviors and can be leveraged to guide NTS training of trauma physicians in the future. More studies are needed to apply these methods to capture NTS from a larger sample of teams in both simulated and real trauma environments.

Figure 1:

Box plot for TTFF on AOI Groups *: p ≤ 0.05; **: p ≤ 0.01; ***: p ≤ 0.001

Figure 2:

Box plot for AFD on AOI Groups

Figure 3:

Box plot for TPS fixated on each AOI *: p ≤ 0.05; **: p ≤ 0.01; ***: p ≤ 0.001