Using natural language processing to compare task-specific verbal cues in coached versus noncoached cardiac arrest teams during simulated pediatrics resuscitation

Abstract

Objectives: Coaches improve cardiopulmonary (CPR) outcomes in real-world and simulated settings. To explore verbal feedback that targets CPR quality, we used natural language processing (NLP) methodologies on transcripts from a published pediatric randomized trial (coach vs. no coach in simulated CPR). Study objectives included determining any differences by trial arm in (1) overall communication and (2) metrics over minutes of CPR and (3) exploring overall frequencies and temporal patterns according to degrees of CPR excellence.

Methods: A human-generated transcription service produced 40 team transcripts. Automated text search with manual review assigned semantic category; word count; and presence of verbal cues for general CPR, compression depth or rate, or positive feedback to transcript utterances. Resulting cue counts per minute (CPM) were corresponded to CPR quality based on compression rate and depth per minute. CPMs were compared across trial arms and over the 18 min of CPR. Adaptation to excellence was analyzed across four patterns of CPR excellence determined by k-shape methods.

Results: Overall coached teams experienced more rate-directive, depth-directive, and positive verbal cues compared with noncoached teams. The frequency of coaches' depth cues changed over minutes of CPR, indicating adaptation. In coached teams, the number of depth-directive cues differed among the four patterns of CPR excellence. Noncoached teams experienced fewer utterances by type, with no adaptation over time or to CPR performance.

Conclusion: NLP extracted verbal metrics and their patterns in resuscitation sessions provides insight into communication patterns and skills used by CPR coaches and other team members. This could help to further optimize CPR training, feedback, excellence, and outcomes.

FIGURE 1

Visualization of NLP methods applied to the data set. Flow diagram of each step in the NLP pipeline with data set examples for word lists and for each unique utterance label. NLP, natural language processing

FIGURE 2

Plots of depth and rate excellence and counts of directive and positives cues over 18‐min CPR sessions by trial arm. Control arm is solid line and intervention arm (coached) is dashed line. Top two plots are average percentage of correct rate (left) and depth (right) plotted per minute of CPR. Middle two plots are total counts of rate directive (left) and depth directive cues (right) plotted per minute of CPR. Bottom two plots are total counts of rate positive (left) and depth positive (right) cues plotted per minute of CPR

FIGURE 3

The average percentage of excellent CPR per minute plotted per tier of CPR excellence determined by k‐shape clustering. Error bars are overlayed per line

FIGURE 4

Mean depth‐directive utterance counts for CPR excellence tiers split by trial arm. Plots of the mean depth‐directive cues per minute of CPR across both trial arms (top plot is controls, bottom is coached) split by tier of excellent CPR determined by k‐shape clustering (as in figure legend). Note that the coached arm had no teams within the worst tier of CPR excellence