Neural substrates of shared visual experiences: a hyperscanning fMRI study

Abstract

Sharing experience is a fundamental human social cognition. Since visual experience is a mental state directed toward the world, we hypothesized that sharing visual experience is mediated by joint attention (JA) for sharing directedness and mentalizing for mental state inferences. We conducted a hyperscanning functional magnetic resonance imaging with 44 healthy adult volunteers to test this hypothesis. We employed spoken-language-cued spatial and feature-based JA tasks. The initiator attracts the partner's attention by a verbal command to a spatial location or an object feature to which the responder directs their attention. Pair-specific inter-individual neural synchronization of task-specific activities was found in the right anterior insular cortex (AIC)-inferior frontal gyrus (IFG) complex, the core node of JA and salience network, and the right posterior superior temporal sulcus, which represents the shared categories of the target. The right AIC-IFG also showed inter-individual synchronization of the residual time-series data, along with the right temporoparietal junction and dorsomedial prefrontal cortex-the core components for mentalization and the default mode network (DMN). This background synchronization represents sharing the belief of sharing the situation. Thus, shared visual experiences are represented by coherent coordination between the DMN and salience network linked through the right AIC-IFG.

Keywords: hyperscanning fMRI; intentionality; joint attention; mentalizing; mirroring system.

Fig. 1.

Experimental setup. (A) The feature-based JA condition. In the Role phase, the participant that is looking at the screen with a cyan frame is the initiator, and the participant with a magenta frame is the responder. The objects have four dimensions such as number (1, 2, 3 or 4), shape (star, heart, circle or square), color (red, blue, yellow or green) and pattern (stripe, border, check or dot). The initiator freely chooses one of the four dimensions (2.5 s). After the frame disappears (the Cue phase), IJA informs the chosen aspect by utterance and the responder says ‘Roger’ in the same period (2.5 s). In the Response and Verify phase, the responder replies with the characteristic of this dimension, and the initiator judges and feeds back, verbally, ‘correct’ or ‘incorrect’ (2.5 s). (B) The sJA condition: The gray four target objects (star, heart, circle or square) are displayed on above, below, left and right from the center of the screen. (C) The feature-based and spatial attention control (solo) condition (fCTRL or sCTRL): The frame color was gray, and participants perform this task without reference to their partners. Brain activity in the Role phase and Cue phase (surrounded by a red frame) was analyzed.

Fig. 2.

The items of feature-based JA. The objects have four dimensions such as number (1, 2, 3 or 4), shape (star, heart, circle or square), color (red, blue, yellow or green) and pattern (stripe, border, check or dot).

Fig. 3.

Inter-brain synchronization of the brain activity during the JA task. Task-specific synchronization (beta synchronization, red) and residual synchronization (blue) are superimposed on the template anatomical MRI scan. Their overlap (magenta) is seen in the right AIC. The plot shows standardized correlation value (z-score) of the residual time-series synchronization during the JA task condition (feature-based and spatial JA) and control condition in the peak voxels of the dmPFC (x = 6, y = 41, z = 35), TPJ (x = 54, y = −43, z = 29) and the AIC-IFG (x = 48, y = 20, z = 5; S3). Error bars indicate the standard error of the mean.

Fig. 4.

Task-related activation with the IJA- RJA gradation within the main effect of JA regions. IJA-dominant regions are shown in red, RJA-dominant regions in blue and common regions in green.

Fig. 5.

Task-related activation with object specificity. Feature-specific activation (green) and spatial location-specific activation (yellow) are superimposed on the template anatomical MRI scan.