The Prediction of Impact of a Looming Stimulus onto the Body Is Subserved by Multisensory Integration Mechanisms

Abstract

In the jungle, survival is highly correlated with the ability to detect and distinguish between an approaching predator and a putative prey. From an ecological perspective, a predator rapidly approaching its prey is a stronger cue for flight than a slowly moving predator. In the present study, we use functional magnetic resonance imaging in the nonhuman primate, to investigate the neural bases of the prediction of an impact to the body by a looming stimulus, i.e., the neural bases of the interaction between a dynamic visual stimulus approaching the body and its expected consequences onto an independent sensory modality, namely, touch. We identify a core cortical network of occipital, parietal, premotor, and prefrontal areas maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus on the faces compared with the activations observed for spatially or temporally incongruent tactile and dynamic visual cues. These activations reflect both an active integration of visual and tactile information and of spatial and temporal prediction information. The identified cortical network coincides with a well described multisensory visuotactile convergence and integration network suggested to play a key role in the definition of peripersonal space. These observations are discussed in the context of multisensory integration and spatial, temporal prediction and Bayesian causal inference.SIGNIFICANCE STATEMENT Looming stimuli have a particular ecological relevance as they are expected to come into contact with the body, evoking touch or pain sensations and possibly triggering an approach or escape behavior depending on their identity. Here, we identify the nonhuman primate functional network that is maximally activated by tactile stimulations presented at the predicted time and location of impact of the looming stimulus. Our findings suggest that the integration of spatial and temporal predictive cues possibly rely on the same neural mechanisms that are involved in multisensory integration.

Keywords: fMRI; looming visual; macaque; multisensory integration; prediction; tactile.

Figure 1.

Methods. A, Visual stimuli consisted in a video sequence of a cone placed in a 3D environment and looming toward the animal's face. The red dot corresponds to the spatial location the monkey was required to fixate to be rewarded. B, The trajectory of the looming cone could start from four different points in the back of the visual scene, four ipsilateral and four contralateral to the predicted impact location with respect to the monkey's face. C, Tactile stimulations were directed to the center of the face thanks to airpuffs directed to the left or right cheek, coinciding with the predicted impact of the looming visual stimulus on the monkey's face. D, We used a mixed fMRI design. Each run was thus composed of five conditions organized in blocks (15 pulses per condition), during which visual (looming cone) and tactile (50 ms airpuff) events were organized as follows: a unimodal visual stimulation condition (V), a unimodal tactile stimulation condition (T), a bimodal condition in which the visual stimulus is spatially and temporally predictive of the tactile stimulus (VT_Full), a bimodal condition in which the visual stimulus predicting the tactile stimulus spatially but preceding the expected impact in time (V_Asynch) and a bimodal condition in which the visual stimulus predicting the tactile stimulus temporally but is spatially incongruent with the tactile stimulus (VT_Incong).

Figure 2.

A, Pupil size changes as a function of the predictive structure of the stimulation blocks. Average normalized changes in pupil size with respect to session average (in %), for each of Monkeys M1 and M2, for the unimodal visual stimulation condition (visual), the unimodal tactile stimulation condition (tactile), the bimodal condition in which the visual stimulus is spatially and temporally predictive of the tactile stimulus (VT_Full), the bimodal condition in which the visual stimulus is temporally asynchronous with the tactile stimulus (VT_Asynch) and the bimodal condition in which the visual stimulus is temporally predictive but spatially incongruent with the tactile stimulus (VT_Incong). The statistical significance of paired t tests is represented as follows: *p < 0.05, **p < 0.01, ***p < 0.001. B, Single run example of eye deviation from the fixation point in distance in degrees, for Monkeys M1 and M2. C, Mean of saccade and microsaccade duration in seconds over all the runs (±median SE), for each type of block and each monkey.

Figure 3.

Whole-brain activation maps for the visual (red), tactile (green), VT temporally asynchronous (blue), VT spatially incongruent (orange), and VT fully predictive (purple) conditions. Each contrast is performed with a level of significance set at p < 0.001 uncorrected level, t > 3.1 for each monkey (M1 and M2). Dark lines represent VT bimodal cortical activations when the time reference is based on the tactile stimulus time rather than on the end of the visual looming sequence, as in the main analysis. 2, Somatosensory area 2; 3b, somatosensory area 3b; 8, area 8; 9/46, area 9/46; 11, area 11; 12, area 12; 13, orbitofrontal area 13; 14, area 14; 23, area 23; 24ab, area 24ab; 24c, area 24c; 24d, area 24d; 32, area 32; 44, area 44; 45, area 45; 46v, ventral area 46; 46d, posterior subdivision of area 46; F4, frontal area F4; FST, floor of superior temporal sulcus; IPa, intraparietal sulcus associated area; IPro, insular proisocortex; LIP, lateral intraparietal area; LOP, lateral occipital parietal cortex; MIP, medial intraparietal area; MST, medial superior temporal area; MT, middle temporal area; PECg, parietal area PE, cingulate part; PGm, parietal area PG, medial part; PIP, posterior intraparietal area; PMZ, premotor zone; Ri, retroinsular area; S2, secondary somatosensory cortex; ST2g, superior temporal sulcus area 2, gyral part; TEa, temporal area TEa; TEO, temporal area TE, occipital part; TPo, temporal parieto-occipital associated area in sts; V1, visual area 1; V2, visual area 2; V2v, ventral visual area 2; V2d, dorsal visual area 2; V3a, visual area 3a; V3, visual area 3; V4, visual area 4.

Figure 4.

Whole-brain activation maps for the VT fully predictive versus fixation contrast masked by the VT temporally asynchronous versus fixation contrast (left; p < 0.05) and the VT spatially incongruent versus fixation contrast (right; p < 0.05) for Monkey M1 (top flat maps) and Monkey M2 (bottom flat maps). All else as in Figure 3.

Figure 5.

Temporal and spatial prediction of imminence of impact maximizes cortical activations. ROIs are defined on the VT fully predictive condition time-locked to the end of the visual looming sequence versus fixation contrast in the two monkeys, using half of the available runs (M1: 10 ROIs; M2: 13 ROIs). PSC is calculated onto the remaining independent half of the runs. A, PSC in the VT fully predictive condition as a function of the PSC in the VT temporally asynchronous condition (one-way ANOVA; M1: p < 0.001, M2: p < 0.001). B, PSC in the VT fully predictive condition as a function of the PSC in the VT spatially incongruent condition (one-way ANOVA; M1: p < 0.002, M2: p < 0.001).

Figure 6.

Impact prediction activates a parietofrontal network. Histograms represent the percentage signal change for visual (red), tactile (green), VT fully predictive (purple), VT spatially incongruent (orange) and VT temporally asynchronous (blue) conditions for Monkeys 1 and 2, for selected ROIs in the extrastriate cortex (MST), the parietal cortex (area VIP) and the premotor cortex (area F4). As in Figure 5, the ROIs are selected on a VT predictive versus fixation contrast using half of the available runs (p < 0.001, uncorrected level), and the extraction of the PSC is performed onto the remaining independent half of the runs. For each ROI, block effect is assessed by a repeated measure one-way ANOVA; significance of PSCs difference with respect to baseline and among themselves is assessed using paired t tests. *p < 0.05, **p < 0.01, ***p < 0.001; °0.05 < p < 0.07.