Less is more: Removing a modality of an expected olfactory-visual stimulation enhances brain activation

Abstract

In recent years, multisensory integration of visual and olfactory stimuli has extensively been explored resulting in the identification of responsible brain areas. As the experimental designs of previous research often include alternating presentations of unimodal and bimodal stimuli, the conditions cannot be regarded as completely independent. This could lead to effects of an expected but surprisingly missing sensory modality. In our experiment, we used a common functional magnetic resonance imaging (fMRI) study design with alternating strong unimodal and bimodal olfactory-visual food stimuli, in addition to a slight overhang of the bimodal stimuli in an effort to examine the effects of removing a visual or olfactory congruent stimulus for older people (41-83 years). Our results suggest that the processing of olfactory and visual stimuli stays intact over a wide age-range and that the utilization of strong stimuli does not lead to superadditive multisensory integration in accordance with the principle of inverse effectiveness. However, our results demonstrate that the removal of a stimulus modality leads to an activation of additional brain areas. For example, when the visual stimulus modality is missing, the right posterior superior temporal gyrus shows higher activation, whereas the removal of the olfactory stimulus modality leads to higher activation in the amygdala/hippocampus and the postcentral gyrus. These brain areas are related to attention, memory, and the search of the missing stimulus. Consequently, careful attention must be paid to the design of a valid, multimodal sensory experiment while also controlling for cognitive expectancy effects that might confound multimodal results.

Keywords: expectations; fMRI; multisensory; olfaction; stimulus removal; vision.

FIGURE 1

Experimental design. A yellow fixation cross (duration 300 ms) signals the arrival of sensory stimulation. For 3 s, participants were presented an odor or no odor and saw or did not saw a congruent picture (for the different conditions, see Figure 2). After stimulus presentation, participants had 4.5 s to rate odor pleasantness on a visual analog scale. A black fixation cross with a jittered duration of 8–12 s separated one trial from the next trial

FIGURE 2

Experimental stimulus combinations and resulting conditions. Each combination of attractive vs. aversive with visual vs. olfactory was presented eight times per session. In addition, an ambiguous bimodal combination was also presented eight times per session. This combination was not included in our analysis, but severe to evoke the expectation of a bimodal representation in the subject without familiarization to a special combination. Resulting conditions combined aversive and attractive stimuli to get pleasantness‐independent results

FIGURE 3

Boxplot of pleasantness ratings of the presented odors collapsed across all participants

FIGURE 4

Interaction plots for three respective factors modality, pleasantness category and age group. ns: nonsignificant, *p < .05, **p < .01

FIGURE 5

Significant brain activations (p < 0.05 FWE whole‐brain comparison, extended cluster threshold of k = 10) if unimodal and baseline conditions are compared with the bimodal condition. A missing olfactory stimulus (green) leads to activations in the left hemisphere, especially in postcentral (cluster (C) 3) and superior temporal gyrus (C6) and the hippocampus (C8). In addition, the cuneus (C2) was significantly activated. A missing olfactory (green) as well as a missing visual stimulus (red) lead to activations in brain regions adjacent to the right supramarginal and angular gyri (C4 and C5). If both stimuli were missing (baseline condition, blue), the BOLD signal in left hemisphere regions was significantly enhanced. Clusters labeled according to the AAL3 atlas (Rolls et al., 2020): C1 L. inferior parietal sulcus, C2 L. cuneus, C3 L. postcentral gyrus, C4 R. rolandic operculum adjacent to r. angular and supramarginal gyrus, C5 R superior temporal gyrus adjacent to R. angular and supramarginal gyrus, C6 L. superior temporal gyrus, C7 L. parahippocampal gyrus, C8 L. hippocampus (L. middle temporal gyrus). For detailed information, see Table 1

FIGURE 6

Beta estimates of clusters of brain areas that were significantly more activated in unimodal or baseline conditions than in the bimodal condition. V, O, OV, and BL are the conditions

FIGURE 7

Schematic overview of processes in the brain if an expected stimulus is missing. Due to priming or experience, expectations are generated. If a stimulus modality is missing, this could lead to effects of an unfulfilled expectation, to the search of the missing stimulus or an attention shift. As a result, additional brain areas related to memory, search or attention are activated