Physiological Synchrony in EEG, Electrodermal Activity and Heart Rate Detects Attentionally Relevant Events in Time

Abstract

Interpersonal physiological synchrony (PS), or the similarity of physiological signals between individuals over time, may be used to detect attentionally engaging moments in time. We here investigated whether PS in the electroencephalogram (EEG), electrodermal activity (EDA), heart rate and a multimodal metric signals the occurrence of attentionally relevant events in time in two groups of participants. Both groups were presented with the same auditory stimulus, but were instructed to attend either to the narrative of an audiobook (audiobook-attending: AA group) or to interspersed emotional sounds and beeps (stimulus-attending: SA group). We hypothesized that emotional sounds could be detected in both groups as they are expected to draw attention involuntarily, in a bottom-up fashion. Indeed, we found this to be the case for PS in EDA or the multimodal metric. Beeps, that are expected to be only relevant due to specific "top-down" attentional instructions, could indeed only be detected using PS among SA participants, for EDA, EEG and the multimodal metric. We further hypothesized that moments in the audiobook accompanied by high PS in either EEG, EDA, heart rate or the multimodal metric for AA participants would be rated as more engaging by an independent group of participants compared to moments corresponding to low PS. This hypothesis was not supported. Our results show that PS can support the detection of attentionally engaging events over time. Currently, the relation between PS and engagement is only established for well-defined, interspersed stimuli, whereas the relation between PS and a more abstract self-reported metric of engagement over time has not been established. As the relation between PS and engagement is dependent on event type and physiological measure, we suggest to choose a measure matching with the stimulus of interest. When the stimulus type is unknown, a multimodal metric is most robust.

Keywords: EEG; electrodermal activity; heart rate; inter-subject correlations; interpersonal physiology; multimodal; physiological synchrony.

FIGURE 1

Illustration of the event detection paradigm. (A) Consider the physiological response traces recorded from N participants who were presented with the same external stimuli at the same time. The timestamps of the data recordings can be separated in moments where events were presented and where an event detection would thus be correct (True) and moments where no events where presented and where an event detection would thus be incorrect (False). (B) Rather than considering the raw physiological responses, the detection paradigm is based on the PS between the participants. (C) Now let us define a threshold t. The moments in time where the synchrony is higher than t are marked as an event (Positive) and the moments in time where the synchrony is lower than t are marked as a non-event (Negative). (D) Rather than using a single value for t, we consider a gradually changing threshold t₀ to t_n, so that at t₀ all data are marked as Positive and at t_n all data are marked as Negative (E) For each iteration t_i, we can now define the true positives (TP), false positives (FP), true negatives (TN), and false negatives (FN) and thus compute the true-positive rate (TPR) and false-positive rate (FPR). (F) Plotting the FPR versus the TPR – both as a function of t – results in the receiver operating curve (ROC). Detection performance is defined as the standard metric of area under the ROC (AUC of ROC).

FIGURE 2

AUC of ROC metric of stimulus detection performance using PS in EEG, EDA, IBI and a multi-modal combination of the three (MM). Performance is shown for the AA and SA groups, when considering only beep blocks or emotional sounds as true positives and when considering both types of stimuli as true positives. In addition, the mean and standard deviation chance level detection performance based on 1,000 renditions with randomized stimulus timing is shown with test results comparing detection performance to chance level (^∗p < 0.05,^∗∗p < 0.01, ^∗∗∗p < 0.001). Note that for the AA group, the emotional sounds but not the beeps are expected to draw (bottom-up) attention, i.e., for the AA group we expect high AUC for emotional sounds only. For the SA group, both beep sequences and emotional sounds are relevant and expected to draw attention.

FIGURE 3

Self-reported engagement scores for audio clips corresponding to moments in the audiobook with high PS (closed markers) or low PS (open markers) in EEG, EDA, IBI and the multimodal metric (MM) (^∗∗p < 0.01, ^∗∗∗p < 0.001).