Neural synchronization during face-to-face communication

Abstract

Although the human brain may have evolutionarily adapted to face-to-face communication, other modes of communication, e.g., telephone and e-mail, increasingly dominate our modern daily life. This study examined the neural difference between face-to-face communication and other types of communication by simultaneously measuring two brains using a hyperscanning approach. The results showed a significant increase in the neural synchronization in the left inferior frontal cortex during a face-to-face dialog between partners but none during a back-to-back dialog, a face-to-face monologue, or a back-to-back monologue. Moreover, the neural synchronization between partners during the face-to-face dialog resulted primarily from the direct interactions between the partners, including multimodal sensory information integration and turn-taking behavior. The communicating behavior during the face-to-face dialog could be predicted accurately based on the neural synchronization level. These results suggest that face-to-face communication, particularly dialog, has special neural features that other types of communication do not have and that the neural synchronization between partners may underlie successful face-to-face communication.

Figure 1.

Experimental procedures. A, Face-to-face communication. B, Back-to-back communication. C, The first optode probe set placed on the frontal, temporal, and parietal cortices. D, The second optode probe set placed on the dorsal lateral prefrontal cortex.

Figure 2.

Neural synchronization increase. t-maps are for the original pairs (A–D), random pairs (E–H), face-to-face dialog (A, E), face-to-face monologue (B, F), back-to-back dialog (C, G), and back-to-back monologue (D, H). The warm and cold colors indicate increases and decreases in neural synchronization, respectively. The black rectangle highlights CH3, showing a significant increase in neural synchronization during the face-to-face dialog.

Figure 3.

Contributions of nonverbal cues and turn-taking to the neural synchronization during the face-to-face and back-to-back dialogs. A, Statistical comparisons between SI and SDI. Error bars indicate SE. **p < 0.01. B, Distribution of SI (yellow points) across the entire time course of coherence values in a randomly selected pair of participants. C, A portion of the time course and the corresponding video images recorded during the experiment. The dialog content at that point is transcribed in blue. R and L, Right and left persons, respectively. The type of communication behavior is indicated in black below the image.

Figure 4.

Prediction accuracy for communication behavior based on the neural synchronization of CH3 during face-to-face dialog. Each black point denotes one pair of participants. The dashed line indicates the chance level (0.5). Error bars are SE. ***p < 0.001.