Having a chat and then watching a movie: how social interaction synchronises our brains during co-watching

Abstract

How does co-presence change our neural experience of the world? Can a conversation change how we synchronise with our partner during later events? Using fNIRS hyperscanning, we measured brain activity from 27 pairs of familiar adults simultaneously over frontal, temporal and parietal regions bilaterally, as they co-watched two different episodes of a short cartoon. In-between the two episodes, each pair engaged in a face-to-face conversation on topics unrelated to the cartoon episodes. Brain synchrony was calculated using wavelet transform coherence and computed separately for real pairs and shuffled pseudo) pairs. Findings reveal that real pairs showed increased brain synchrony over right Dorso-Lateral Pre-Frontal cortex (DLPFC) and right Superior Parietal Lobe (SPL), compared to pseudo pairs (who had never seen each other and watched the same movie at different times; uncorrected for multiple comparisons). In addition, co-watching after a conversation was associated with greater synchrony over right TPJ compared to co-watching before a conversation, and this effect was significantly higher in real pairs (who engaged in conversation with each other) compared to pseudo pairs (who had a conversation with someone else; uncorrected for multiple comparisons). The present study has shed the light on the role of social interaction in modulating brain synchrony across people not just during social interaction, but even for subsequent non-social activities. These results have implications in the growing domain of naturalistic neuroimaging and interactive neuroscience.

Keywords: Brain-to-brain synchrony; cowatching; fNIRS hyperscanning; social interaction; wavelet coherence.

Figure 1

Inter-subject correlation vs Brain synchrony. Left: on day 1, one participant is watching a cartoon while receiving a brain scan. On day 2, another participant is undergoing the same procedure, watching the same cartoon and receiving a brain scan. The neural responses is than compared across different participants who did the same task at separate times and on their own. The similarity across neural responses to the same stimulus (e.g. the cartoon) is computed as inter-subject correlation. Right: On the same day, two participants watch a cartoon together, in real time, next to each other. Their brains response is recorded simultaneously, and the coherence between the two signals is computed as brain-to-brain synchrony.

Figure 2

Example of data processing streamline for one dyad (and one channel/ROI) – A. participants seat next to each other and watch an episode of the BBC series ‘Dipdap’ (Phase 1). After watching one episode, the two participants engage in a social interaction task, when they chat about unrelated topics (Phase 2). They than watch another – novel - episode of Dipdap (Phase 3). During co-watching (phase 1 and 3), a separator ensures that participants do not engage in any form of communication. The two Dipdap episodes were randomly allocated to phase 1 or 3 (counterbalanced across dyads). They are all non-verbal, self-contained, identical for duration and comparable in terms of audio/visual features. B. Full session Nirs Signal (HbCBSI) plotted for participant A (red) and participant B (blue). Nirs signal during each video cowatching is highlighted. C. Wavelet coherence spectrogram for video 1 and video 2. Bars show the frequency of interest used in analysis. D. Bars plot of the mean for the three frequencies of interest (High: 0.1-0.2 Hz, Medium: 0.03-0.1 Hz, Low: 0.02-0.03 Hz) for video 1 and video 2. Data plotted in B., C. and D. belongs to the same dyad.

Figure 3

From headset probe locations to Region of Interest – A. NIRS headset configuration. Optodes are divided by 7 sources and 7 detectors per hemisphere, spreading from parietal to frontal regions. This configuration forms 19 channels per hemisphere, for a total of 38 channels per participant. B. Channel localization in standard space. Channels (1-38) are plotted, each assigned to one colour, over the whole sample. C. 8 functional ROIs are plotted in yellow, 4 in each hemisphere. Green dots are channels for one participant. For each ROI, the closest channel to the centre would be assigned and contribute with data. No more than one channel would contribute to each ROI per participant. To be assigned to an ROI, channels must be located within the area marked by the dark yellow dotted line around that ROI centre (radius 2cm). D. Channels plotted after being assigned to one of the 8 ROIs. Each colour represent one ROI. DLPF = Dorso-Lateral Pre-Frontal cortex; VPMC = Ventral Pre-Motor cortex; TPJ = Temporo-Parietal Junction; SPL = Superior Parietal Lobe.

Figure 4

Data analysis pipeline. Schematic of the data analysis pipeline for one dyad. Step 1: data is collected from a real dyad (two people visiting the lab together); Step 2: nirs signal from one session is split between the participant-red‘s signal and the participant-blue‘s signal; Step 3: nirs signal for each channel goes through pre-processing and visual quality checks (see methods), after which it is either included or excluded; Step 5: trials of interest (e.g. ‘Movie 1’, see Methods) are extracted from the full session timeseries; Step 5: wavelet coherence analysis is run between participants separately for each trial to obtain a measure of brain synchrony during that trial. Step 6: measures of interest are computed, namely i) coherence during co-watching movie 1 and ii) coherence change after the conversation phase (coherence co-watching movie 2 minus coherence co-watching movie 1). Step 4, 5 and 6 are also identically executed for pseudo dyads (*). Pseudo dyads are computed on the basis of some pre-assigned characteristics to match real dyads on all experimental factors (e.g. trial order, participant colour allocation, see Methods). Step 6: 10,000 permutations were run for each measure of interest separately, between values obtained from the real dyads and values obtained from the pseudo dyads.

Figure 5

Results for brain coherence during movie co-watching (phase 1) for real and pseudo dyads. Boxplots showing the distribution for real dyads (yellow) and for pseudo dyads (purple) of the brain coherence during co-watching phase 1 (before conversation). There was significantly more coherence (low frequency 0.02-0.03 Hz) in real vs pseudo dyads over right DLPFC (right panel) and right SPL (left panel). DLPFC = dorso-lateral pre-frontal cortex; SPL = superior parietal lobe.

Figure 6

Results for brain coherence change after engaging in conversation. Plots of brain coherence difference between co-watching 2 (post-conversation) and co-watching 1 (pre-conversation). Left panel: boxplots of the distribution of brain coherence difference across co-watching phases for real (yellow) and pseudo (purple) dyads. After a conversation, there was significantly more coherence in real vs pseudo dyads over right TPJ. Right panel: brain coherence for real dyads (sample mean) during co-watching preconversation (phase 1) and co-watching post-conversation (phase 2) over session duration. ^*p<.05. SPL = Superior Parietal Lobe, TPJ = Temporo-Parietal Junction.