The spatiotemporal dynamics of illusory contour processing: combined high-density electrical mapping, source analysis, and functional magnetic resonance imaging

Abstract

Because environmental information is often suboptimal, visual perception must frequently rely on the brain's reconstruction of contours absent from retinal images. Illusory contour (IC) stimuli have been used to investigate these "filling-in" processes. Intracranial recordings and neuroimaging studies show IC sensitivity in lower-tier area V2, and to a lesser extent V1. Some interpret these data as evidence for feedforward processing of IC stimuli, beginning at lower-tier visual areas. On the basis of lesion, visual evoked potentials (VEP), and neuroimaging evidence, others contend that IC sensitivity is a later, higher-order process. Whether IC sensitivity seen in lower-tier areas indexes feedforward or feedback processing remains unresolved. In a series of experiments, we addressed the spatiotemporal dynamics of IC processing. Centrally presented IC stimuli resulted in early VEP modulation (88-100 msec) over lateral-occipital (LOC) scalp--the IC effect. The IC effect followed visual response onset by 40 msec. Scalp current density topographic mapping, source analysis, and functional magnetic resonance imaging results all localized the IC effect to bilateral LOC areas. We propose that IC sensitivity described in V2 and V1 may reflect predominantly feedback modulation from higher-tier LOC areas, where IC sensitivity first occurs. Two additional observations further support this proposal. The latency of the IC effect shifted dramatically later (approximately 120 msec) when stimuli were laterally presented, indicating that retinotopic position alters IC processing. Immediately preceding the IC effect, the VEP modulated with inducer eccentricity--the configuration effect. We interpret this to represent contributions from global stimulus parameters to scene analysis. In contrast to the IC effect, the topography of the configuration effect was restricted to central parieto-occipital scalp.

Fig. 1.

Stimuli and experimental paradigm.Left, Kanisza-type inducers were used in all experiments of the present study to define various geometric shapes (see Materials and Methods and Table 1 for details). Inducers were oriented to either form or not form an illusory shape. Right, The timing of stimulus presentations was such that inducers were presented for 500 msec duration (400 msec in experiment 5), followed by a blank screen for 1000 msec (450 msec in experiment 5). This was followed in turn by a Y/N prompt that remained on the screen until subjects made a forced-choice button-press response indicating whether an illusory shape was presented. A blank screen (1000 msec duration in experiments 1, 3, and 4; 700 msec duration in experiment 5) then preceded the next trial. Paradigmatic differences in experiment 2 are described in Materials and Methods.

Fig. 2.

Experiment 1 VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off). Data from a pair of frontal (F3/F4) and parieto–occipital (PO5/PO6) electrode sites are shown. Their locations on the scalp are indicated by large white discs on the 3-D reconstruction (BEM as implemented in CURRY) of one subject's (B.H.) anatomical MRI. Black tracesindicate the VEP response to the presence of an illusory contour (IC present), whereas light gray tracesindicate the corresponding VEP response to the non-inducing configurations (IC absent). Dark gray dashed traces represent the IC present minus IC absent difference.Black traces in the insets illustrate thep value of point-wise t tests between the IC present and IC absent conditions across the VEP epoch.

Fig. 3.

SCD topographic maps of the IC effect for centrally presented stimuli. Maps in this and similar figures are displayed on the 3-D reconstruction (BEM as implemented in CURRY) of one subject's (B.H.) anatomical MRI data. These SCD foci are consistent with bilateral lateral–occipital generators, although they are more pronounced over the right hemisphere. Polarity of these maps is arbitrary, depending on the direction of the subtraction, and scales are shown. A, SCD topographic map (left-sided, back, and right-sided views) at 146 msec after stimulus onset depicting the IC effect in experiment 1 (inducers appeared black on agray background). B, SCD topographic map at 156 msec after stimulus onset depicting the IC effect in experiment 3 (inducers appeared gray on a black background; identical scale as in A).C, A series of maps (back view) depicting the stability of the SCD topography over the 88–168 msec post-stimulus epoch for the IC effect shown in A.

Fig. 4.

A, The positions and orientations of two fixed dipoles (cyan) are rendered in the 3-D reconstruction of one subject's (B.H.) anatomical MRI (BEM as implemented in CURRY; back and side views shown) at the peak of the IC effect (146 msec). B, On average, this pair of dipoles accounts for 95.1% of the variance between the observed data and the forward solution to these dipoles over the 116–156 msec post-stimulus epoch. C, The strength of these dipoles over the post-stimulus epoch indicates synchronous bilateral IC processing in the lateral–occipital areas.

Fig. 5.

VEP and fMRI results from experiment 2.A, VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off; identical color scheme as in Fig. 2) from two representative scalp sites illustrate the IC effect under passive viewing conditions. Electrode locations are indicated with large white discson the 3-D scalp reconstruction of one subject's anatomical MRI shown in the insets. B, The locations of fMRI results are shown on axial slices of a standard brain supplied with SPM99 software. White pixels indicate areas of significant BOLD signal increase for the IC present versus IC absent conditions (p ≤ 0.05; corrected for multiple comparisons across the entire image volume).

Fig. 6.

VEP/fMRI co-registration and source analysis.A, The locations of the fMRI activation clusters (yellow) and VEP dipoles (blue), both from analyses of the group (n = 5) data of experiment 2, are shown spatially co-registered on the 3-D reconstruction of one subject's anatomical MRI. B, On average, these dipoles account for 87.7% of the variance between the observed data and the forward solution to these dipoles over the 106–146 msec post-stimulus epoch. C, The strength of these dipoles over the 80–300 msec post-stimulus epoch reveals the relative contribution of each source over time.

Fig. 7.

Experiment 3 VEP waveforms (40 Hz low-pass filter; 24 dB/octave roll-off). Data are shown in an identical manner as in Figure 2 from two representative electrode sites (PO5and PO6).

Fig. 8.

Shape-wise analyses demonstrating the configuration effect. A, Bar graphs display the mean area (66–86 msec after stimulus) of the VEP response (sitesP7, Pz, and P8) to each shape independent of IC presence versus absence. A step function in mean area is observed over midline but not lateral scalp sites. Asingle asterisk indicates a significant difference (p < 0.01; paired t test) in mean area between both the response to the square and triangle versus a particular shape. The double asterisk indicates a significant difference (p < 0.01; pairedt test) in mean area between the circle and each of the other shapes. Mean area would appear to follow the eccentricity of inducers. B, A diagram of each of the IC shapes (thick black lines) indicates the variation in the farthest eccentricity of any inducer across shapes. C, VEP waveforms from sites P7, Pz, andP8. These data have been collapsed acrosswide shapes (square and triangle) andnarrow shapes (star, pentagon, and circle).D, SCD topographic maps of the wide − narrow difference at 76 msec after stimulus onset.

Fig. 9.

Experiment 5 VEP waveforms in response to left and right visual field presentations (40 Hz low-pass filter; 24 dB/octave roll-off). Data are shown in an identical manner as in Figure 2 from one contralateral representative electrode site for left visual field (PO6) and right visual field (PO5) stimulus presentations.

Fig. 10.

SCD topographic maps (left-sided, back, and right-sided views) of the IC effect for laterally presented stimuli of experiment 5. Top, SCD topographic maps at 236 msec after stimulus onset depicting the IC effect in response to left visual field stimuli. Bottom, SCD topographic maps at 236 msec after stimulus onset depicting the IC effect in response to right visual field stimuli. These SCD foci are consistent with bilateral lateral–occipital generators, although more pronounced over the contralateral hemisphere. Polarity of these maps is arbitrary, depending on the direction of the subtraction, and scales are shown.

Fig. 11.

Statistical cluster plots. Color values indicate the result of point-wise t tests evaluating the IC effect across post-stimulus time (x-axis) and electrode positions (y-axis) for 55 of the 64 electrodes (see Materials and Methods for details of electrode locations) used in experiments 1, 3, and 5. For clarity, only pvalues < 0.01 are color encoded. Top, Results from experiments 1 and 3 using centrally presented inducers (left, gray background;right, black background) indicate a biphasic IC effect over posterior scalp sites. No robust IC effect is observed over frontal sites. Bottom, Results from experiment 5 using laterally presented inducers indicate that the IC effect observed with centrally presented inducers shifts ∼120 msec later. As with centrally presented stimuli, no IC effect is observed over frontal sites. The lag in onset of the IC effect over the direct (contralateral) and indirect (ipsilateral) hemispheres can be readily seen for both left and right visual field presentations.