Dissociating the effect of noise on sensory processing and overall decision difficulty

Abstract

It has been proposed that perceptual decision making involves a task-difficulty component, which detects perceptual uncertainty and guides allocation of attentional resources. It is thought to take place immediately after the early extraction of sensory information and is specifically reflected in a positive component of the event related potentials, peaking at ∼ 220 ms after stimulus onset. However, in the previous research, neural processes associated with the monitoring of overall task difficulty were confounded by those associated with the increased sensory processing demands as a result of adding noise to the stimuli. Here we dissociated the effect of phase noise on sensory processing and overall decision difficulty using a face gender categorization task. Task difficulty was manipulated either by adding noise to the stimuli or by adjusting the female/male characteristics of the face images. We found that it is the presence of noise and not the increased overall task difficulty that affects the electrophysiological responses in the first 300 ms following stimulus onset in humans. Furthermore, we also showed that processing of phase-randomized as compared to intact faces is associated with increased fMRI responses in the lateral occipital cortex. These results revealed that noise-induced modulation of the early electrophysiological responses reflects increased visual cortical processing demands and thus failed to provide support for a task-difficulty component taking place between the early sensory processing and the later sensory accumulation stages of perceptual decision making.

Figure 1.

Stimuli. The panel on the left shows samples of the female and male face stimuli used in the control condition. By morphing these paired female and male face images, we created stimuli with ambiguous face gender attributes and used these intermediate androgynous face images in the N-a condition (top right). In the N-p condition (bottom right), we used noisy face stimuli, which were created by decreasing the phase coherence of the face images used in the control condition. The intermediate smaller face pairs in the middle of the figure are included to better illustrate the different image manipulations but were not presented during the experiment.

Figure 2.

Behavioral results. A, B, Both accuracy (A) and reaction times (B) were significantly impaired as a result of stimulus modulation both in the EEG and in the fMRI experiment. Error bars indicate ±SEM (N = 16 in both cases, ***p < 0.001).

Figure 3.

Grand-average ERPs and peak amplitudes of the main experiment. A, B, ERPs evoked by the control, morphed, and noisy faces are shown in black, blue, and brown, respectively, alongside the GFP (B) waveforms averaged over subjects. C details the voltage topographies and amplitudes of the peaks P1, N170, P2, and P3b measured over the respective electrode clusters marked with black dots on the topographical maps. In the cases of P1, N1, and P2, the noise-present condition significantly differs from the rest, while in the case of P3b, there is only a nonsignificant trend. Maps are the average of the three conditions at the time point shown below. Note that cartoon heads are plotted with unrealistic head radius for better electrode visibility. Error bars indicate ±SEM (N = 16, **p < 0.01, ***p < 0.001; n.s., not significant).

Figure 4.

Grand-average ERPs of the control experiment. In the case of the gender categorization task, there was a clear modulation of the P2 amplitude with changing the phase coherence of the images (A), while there was no modulation when changing the difficulty of the decision while keeping the phase coherence constant in the tilt discrimination task (B) (N = 14).

Figure 5.

Grand average ITC. A, Spectrogram of a typical electrode (PO8—position indicated by a white dot on cartoon heads) showing the N-a and N-p conditions above and below, respectively. Spectrogram of the C condition was very similar to that of the N-a condition and is not shown. Insets show the ITC topographies averaged over 100–200 ms and 4–7 Hz as indicated by the black boxes. Black dots show the three electrode clusters for which ITC time courses were calculated (B). Black, C; blue, N-a; brown, N-p. C, Correlation between the theta ITC difference between N-p and N-a conditions and the respective amplitude differences of P1, N170, and P2 peaks (N = 16). Noise-induced modulation of the power of the theta band oscillations was analogous to that found in the case of ITC (see supplemental Fig. S3, available at www.jneurosci.org as supplemental material).

Figure 6.

Results of the whole-brain conjunctions and ROI analyses. A, depicts bilateral posterior LOC (DOT-LOS) as the only cortical areas showing higher activation to the noise-present condition relative to both the control and the noise-absent conditions (above) and left insula and right inferior frontal gyrus as the areas with stronger fMRI response to both noise-absent and -present relative to the control condition (below). Maps are displayed with p_unc < 10⁻⁴ on the PALS-B12 partially inflated brain (Van Essen, 2005) and on the averaged anatomical images of all 16 subjects. B, Based on an independent localizer, the ROI analysis in agreement with the whole-brain analysis shows higher bilateral DOT-LOS activation for noise-present condition relative to the other two, while FFA does not show any significant modulation (***p < 0.001; Ins, insula; iFG, inferior frontal gyrus).