Crossmodal processing in the human brain: insights from functional neuroimaging studies

Abstract

Modern brain imaging techniques have now made it possible to study the neural sites and mechanisms underlying crossmodal processing in the human brain. This paper reviews positron emission tomography, functional magnetic resonance imaging (fMRI), event-related potential and magnetoencephalographic studies of crossmodal matching, the crossmodal integration of content and spatial information, and crossmodal learning. These investigations are beginning to produce some consistent findings regarding the neuronal networks involved in these distinct crossmodal operations. Increasingly, specific roles are being defined for the superior temporal sulcus, the inferior parietal sulcus, regions of frontal cortex, the insula cortex and claustrum. The precise network of brain areas implicated in any one study, however, seems to be heavily dependent on the experimental paradigms used, the nature of the information being combined and the particular combination of modalities under investigation. The different analytic strategies adopted by different groups may also be a significant factor contributing to the variability in findings. In this paper, we demonstrate the impact of computing intersections, conjunctions and interaction effects on the identification of audiovisual integration sites using existing fMRI data from our own laboratory. This exercise highlights the potential value of using statistical interaction effects to model electrophysiological responses to crossmodal stimuli in order to identify possible sites of multisensory integration in the human brain.