The timing of feedback to early visual cortex in the perception of long-range apparent motion

Abstract

When 2 visual stimuli are presented one after another in different locations, they are often perceived as one, but moving object. Feedback from area human motion complex hMT/V5+ to V1 has been hypothesized to play an important role in this illusory perception of motion. We measured event-related responses to illusory motion stimuli of varying apparent motion (AM) content and retinal location using Electroencephalography. Detectable cortical stimulus processing started around 60-ms poststimulus in area V1. This component was insensitive to AM content and sequential stimulus presentation. Sensitivity to AM content was observed starting around 90 ms post the second stimulus of a sequence and most likely originated in area hMT/V5+. This AM sensitive response was insensitive to retinal stimulus position. The stimulus sequence related response started to be sensitive to retinal stimulus position at a longer latency of 110 ms. We interpret our findings as evidence for feedback from area hMT/V5+ or a related motion processing area to early visual cortices (V1, V2, V3).

Figure 1.

Stimuli used in experiment I. (a) Stimulus timing and sequences: Each square denotes presentation of the corresponding screen for 200 ms. Black screens with white cross: presentation of the fixation cross alone. Black screen with white square: presentation of a white square in the corresponding location (details of the stimulus geometry can be found in (b)). The red frame denotes the corresponding baseline interval used in the evaluation of the VEPs. All stimulus timestamps are aligned to the onset of the second stimulus in the corresponding AM stimulus condition (I-AM or I-AMweak) to allow a later comparison of the cortical events following the second stimulus in each AM stimulus condition. From top to bottom: (I-AM) AM inducing stimulus condition with presentation of a white square for 200 ms in the upper visual field (position I-U) followed by 200 ms of blank screen and presentation of a second white square (S2) in the lower visual field for 200 ms. (I-U-S1) control stimulus condition corresponding to the timing and location of the first stimulus in the AM stimulus condition I-AM. (I-L-S2) control stimulus condition corresponding to the timing and location of the second stimulus in the AM stimulus condition I-AM. Stimulus conditions I-AM, I-U-S1, and I-L-S2 together formed the “AM” experimental condition. (I-AMweak) stimulus condition consisting of the presentation of a white square for 200 ms in the upper visual field (position I-U) followed by 400 ms of blank screen and presentation of a second white square (S2) in the lower visual field for 200 ms. I-AMweak did induce a weak or no AM percept. (I-U-S1) control stimulus condition corresponding to the timing and location of the first stimulus in the AM stimulus condition I-AM. Note that this was exactly the same stimulus condition as in the second row, however the timestamps and the baseline were shifted to accommodate the need for a comparison to the stimulus condition I-AMweak. (I-L-S2(AMweak)) control stimulus condition corresponding to the timing and location of the second stimulus in the AM stimulus condition I-AMweak . Stimulus conditions I-AMweak, I-U-S1 and I-L-S2(AMweak) together formed the experimental condition “weak AM.” (b) Stimulus geometry: stimuli consisted of white squares of 2° visual angle presented at an eccentricity of 4° visual angle at 25° degrees above the mid line or at 45° below the mid line. (c) Sequence of stimulus conditions in an experimental run. All stimulus conditions were presented in blocks of 6. Interstimulus condition intervals (ITI) within a block and the stimulation free IBIs were chosen at random from one of 4 values: 1000, 1500, 2500, or 3000 ms. These intervals included the baseline of the subsequent stimulus condition. The order of the different stimulus blocks (I-AM, I-AM(weak),…) was also pseudo randomized.

Figure 2.

Stimuli used in experiment II to test for a retinotopic modulation of the SSDWs. Each black frame corresponds to the presentation of the corresponding stimulus display for 200 ms. A red border indicates the use of this frame/time interval as a baseline interval in VEP analysis. The time axis is aligned to the onset of second stimulus in the respective AM stimulus conditions (t = 0). (a) From top to bottom: (II-AMupper) AM inducing stimulus condition with a motion path that lies predominantly in the upper visual field. (II-U-S1) Control stimulus condition that matches the first stimulus of II-AMupper in timing and location. (II-M-S2) Control stimulus condition that matches the second stimulus in II-AMupper in timing and location. (II-AMlower) AM inducing stimulus condition with a motion path that lies exclusively in the lower visual field. (II-M-S1) Control stimulus condition that matches the first stimulus of II-AMlower in timing and location. (II-L-S2) Control Stimulus condition that matches the second stimulus in II-AMlower in timing and location. (b) Stimulus geometry: all stimuli were located on a circle of an eccentricity of 4° visual angle; (II-U) first stimulus in the AM inducing condition II-AMupper, located 45° above the horizontal meridian, (II-M) second stimulus in the AM inducing condition II-AMupper or first stimulus in the AM inducing condition II-AMlower, located 15° below the horizontal meridian. (II-L) Second stimulus in the AM inducing condition II-AMlower, located 75° below the horizontal meridian. Red arrows symbolize the AM paths in the upper and lower condition. The stimulus geometry is chosen such that the representation of the lower motion path between the 2 stimuli (in condition II-AMlower) lies on the upper bank of the calcarine sulcus whereas the representation of the upper motion path between the 2 stimuli (in condition II-AMupper) lies at an opposing position on the lower bank of the calcarine sulcus. (c) Sequence of stimulus conditions in an experimental run. All stimulus condition types were presented in blocks of 6. ITIs within a block were randomized (1800–1920 ms, uniform distribution), each block of 6 was followed by a stimulus free IBI that had a randomized duration (1800–1920 ms, uniform distribution). The order of the different stimulus blocks (I-AM, I-AM(weak),…) was also randomized.

Figure 3.

Algorithm used for identification of SSDW components of the VEP. In order to identify those parts of the VEP that differ when stimuli are presented in close temporal proximity (200–400 ms ISI) we first added the VEPs evoked in the control stimulus conditions (here: I-U-S1, red; I-L-S2; magenta) to obtain a composite VEP (here: I-composite(AM), green) that represents the VEP that is expected when no interaction or sequence effects between the stimuli in the AM sequence are present. This composite VEP is then subtracted from the true VEP elicited by the AM stimulus sequence (I-AM, black). The resulting difference wave reflects sequence effects due to either simple nonlinear addition of the underlying neuronal processes (e.g., due to refractive effects) and or processes related to motion and gestalt (motion path) perception. The graph exemplifies this algorithm with grand average VEPs recorded at electrode POz. The actual analysis was performed at the individual subject level for later cluster randomization analysis using within subjects permutation of conditions. The analysis of differences between AM VEPs (“AM”) and the corresponding sum of the single VEPs (“composite”) focused on the interval from 0 to 200 ms, as indicated by the horizontal black bar in the Diff(AM) display. This interval captured all major early peaks in the SSDW. The horizontal dashed lines indicate the minimum to maximum range of fluctuations in the SSDW that occur before the onset of S2. (Insert): Quality of the VEP recordings: VEP responses to the first stimulus in the AM condition (I-AM) and to the corresponding control stimulus (I-U-S1) with identical position and timing. VEPs in both conditions show a very high degree of similarity and no significant differences were found, indicating a good overall reliability of the measurements. To help the reader, VEP component C1 has been marked on the single stimulus VEP traces and VEP component N1 has been marked on the trace of stimulus I-L-S2. Note that electrode POz is not optimum for the display of P1 components. Note the expected dependence on retinal stimulus position (I-U-S1 vs. I-U-S2), both at early time intervals (C1, 60- to 90-ms post actual stimulus onset, originating in V1, compare Fig. 4) and later time intervals (P1, N1 originating either in dorsal (I-U-S2) or ventral (I-U-S1) extrastriate areas; also compare Figure 9.

Figure 4.

Detection of VEP components generated in V1 (C1 component). (a) Geometry of the control stimuli: upper row—upper control stimulus (I-U-S1), 25° above the horizontal meridian, bottom row—lower control stimulus (I-L-S1), 45° below the horizontal meridian, both stimuli are presented at 4° eccentricity (conf. Fig. 1). Stimulus (I-L-S1) corresponds to the stimulus condition I-L-S2 as it was introduced in Figure 1. Here, however, the baseline for computation of the VEP was 200 ms immediately preceeding the stimulus to enable a comparison to the literature. (b) Scalp topographies at a latency of 84 ms for both stimulus conditions. Although results from previous studies (Di Russo et al. 2002) predict a full reversal of scalp polarity between the stimuli we only see a partial rotation of the topography, perhaps indicating that these stimuli did not activate exactly opposing patches of cortex in V1 in our sample of subjects. (c) VEP responses at electrode POz to the presentations of control stimuli at 25° above the vertical meridian (blue, I-U-S1) and 45° below the vertical meridian (red, I-L-S1). Note the polarity reversal between responses evoked by the 2 stimuli at this electrode with a peak difference at a latency of 84 ms after stimulus onset. The last common data point of the 2 signals before this peak difference was found at 62 ms.

Figure 5.

Grand Average VEPs in experiment I: I-AM condition and sum of VEPs from the corresponding control conditions. VEPs are displayed using BESA's standard 81 electrode system in a pseudo topographical arrangement. The grand average was performed after resampling the interpolated scalp voltage topographies of individual subjects’ 63 electrode recordings to BESA standard 81 electrodes. VEPs were filtered between 0.5 and 100 Hz and are presented after onset of the second stimulus in the AM sequence (t = 0 ms). (I-AM, black)—VEP evoked by the AM stimulus. (I-composite(AM), red)—the composite VEP in the control stimulus conditions. (Diff(AM), green)—the SSDW. (Small insert, upper left) enlargement of these VEPs at electrode POz, which showed the largest peak in the SSDW in the time window from 90 to 110 ms (denoted d103). (Small insert, upper right) enlargement of the VEPs at electrode C3, which showed the highest peak amplitude in the later bilateral positivity of the difference wave (denoted bpos).

Figure 6.

VEP topographies in the AM stimulus condition, the composite VEP from the summed control stimulus conditions, the SSDW and the corresponding cluster level statistics over time after the onset of S2 (t = 0 ms). Scalp voltage topographies represent values at the indicated time points. Statistical maps are averaged over an interval of ±12.5 ms around the indicated time points. (Upper Panel) upper row: scalp topographies of the VEP evoked in the AM stimulus condition (I-AM); middle row: scalp topographies of the summed VEPs in the control stimulus conditions (I-composite(AM)); bottom row: scalp topographies of the SSDW. Lower Panel: Scalp topographies of the SSDW masked by membership in a statistically significant electrode/time pair cluster as revealed by cluster level randomization analysis (P < 0.05). The nonsignificant electrode/time pairs are marked in green (n.s.). Significantly different electrode/time pair clusters are shown with their corresponding scalp voltage topography.

Figure 7.

Comparison of the SSDWs for the AM and the AMweak experimental condition. (a) Scalp maps of statistically significant electrode/time pair clusters resulting from cluster level randomization analysis (P < 0.05) for the SSDWs for the AM sequence in experiment I (Diff(AM), upper row) and the slowed down stimulus sequence (Diff(AMweak), bottom row) which elicited only a weak or no motion percept. Nonsignificant electrode time pairs are masked in green; statistically significant electrode/time cluster are shown with their corresponding scalp voltage topography. Although a negative going significant cluster appeared for both the weak and the strong AM stimulus condition around 100–110 ms, the positive going earlier (90–120 ms, dashed red line) cluster existed only in the strong AM stimulus condition (I-AM). The interval from 90 to 100 ms where a significant part of the SSDW was only found for the strong AM condition is marked by a solid red line. This early positive going difference of the 2 SSDWs (Diff(AM) - Diff(AMweak)) was a classified as a strong trend (P < 0.06) when using cluster randomization analysis with 4 connected electrodes. This positive going difference of the 2 SSDWs was significant at the single electrode level (t-test, P < 0.05) at electrodes: P1, Pz, P2, P3, P6, P8, POz, PO4, PO8, PO10, Oz, O2. (b) Difference waves for the 2 conditions Diff(AM) (green) and Diff(AMweak) (blue) at 2 electrode locations: electrode POz where the peak of the early (90–120 ms), motion sensitive part of the difference wave was found in condition Diff(AM) (green) but not in condition Diff(AMweak) (blue); electrode C3 that was close to one of the peaks of a later (150–200 ms) bilateral positive difference wave common to both conditions. The scalp map to the right displays the electrode locations and data averaged over the interval 160–190 ms in condition Diff(AM).

Figure 8.

Scalp topographies of the SSDWs and the effect of a shifted retinal position of the stimuli in the interval 0- to 180-ms post S2. (a) Nonsignificant electrode/time pairs are marked in green, significant electrode/time pair clusters (P < 0.05) are displayed according to their corresponding scalp voltage. From top to bottom: Diff(AMupper)—Difference between VEP in the condition II-AMupper and the summed VEP of the corresponding control stimuli (II-U-S1 + II-M-S2). Diff(AMlower)—Difference between VEP in the condition II-AMlower and the summed VEPs of the corresponding control stimuli (II-M-S1 + II-U-S2). Diff**—Difference of the SSDWs: Diff**=(Diff(AMupper) − Diff(AMlower)). Significant electrode/time pair clusters of this difference Diff** indicate that a part of Diff(AMupper) and Diff(AMlower) is susceptible to shifts in the retinal position of the stimuli and that this part of the SSDWs arises from retinotopically organized cortex, presumably with quarter field representations (confer Figs 2, 9). Note that this significant difference, especially its positive peak were well localized in time between 110 and 150 ms indicating a transition of processing through retinotopically organized cortices, that started to cease at the later stages of our analysis interval. The red box indicates the time interval where SSDWs had been susceptible to the manipulation of motion energy in experiment I (conf. Fig. 7). Note that this part of the difference waves was not susceptible to a retinal shift of stimulus position, indicating that the early, motion energy sensitive part of the SSDW arises from a piece of cortex with no or a nondiscernible retinotopic organization (like hMT/V5+). (b) Difference waves for the 2 conditions Diff(AMupper) (blue), Diff(AMlower) (red), the difference of differences Diff** (green) and the cluster corrected t-statistics (black, dashed) at 2 electrode locations over early visual cortices: electrode POz and PO7. The time range (80–180 ms) is identical to the one presented in (a). The tvalues (dashed black line) are masked by statistical significance in the same way as the maps in (a). The inserted map shows the sample electrode locations and data from condition Diff** averaged from 130 to 160 ms.

Figure 9.

fMRI activations evoked by the AM stimuli II-AMupper and II-AMlower. (a) Schematic drawing of an inflated left cortical hemisphere seen from a posterior/medial viewpoint. The extent of retinotopically organized cortex relevant for this study is highlighted in light gray. Representations of the horizontal meridian of the visual field are depicted in blue; representations of the vertical meridian are depicted in green. Activations in response to the inducing single stimuli that were expected based on the known retinotopy of early visual areas and previous studies with similar stimuli (Muckli et al. 2005) are depicted as little colored dots (II-U = II-U-S1; II-M = II-MS1 and II-M-S2; II-L = II-L-S2). Note that the expected centers of gravity are depicted, whereas no indication of the actual size is intended here. (b) Actual fMRI activations evoked by the AM stimulus conditions II-AMupper (yellow) and II-AMlower (red) and their overlap (orange). These AM stimulus conditions consisted of the single inducing stimuli II-U-S1/II-M-S2 (II-AMupper) and II-MS1/II-L-S2 (II-AMlower). Activations in a sample subject are depicted on the inflated left cortical hemisphere of this subject. The visual areas V1/V2/V3 V3a and hMT/V5+ were marked based on results from a previous retinotopic mapping experiment (Muckli et al. 2005). Activations in both conditions strongly overlap in visual area hMT/V5+, whereas the ventral visual areas V1v, V2v, Vp, and V4v are only activated by condition II-AMupper. The expected overlap of the 2 conditions due to the shared single stimulus II-M (being either II-M-S1 or II-M-S2) is found exclusively in the dorsal visual areas V1d, V2d, V3d because this stimulus was presented below the horizontal meridian. The positions of stimuli II-M-S1 and II-M-S2 had been chosen to place them in the fundus of the calcarine sulcus in the average subject (Di Russo et al. 2002).

Figure 10.

Group level fMRI results. Group level fMRI activations in conditions II-AMupper (yellow), II-AMlower (red), and their overlap (orange) projected onto the gray-white matter boundary of the Talairach transformed brain of one subject (A.K.). All activated clusters were thresholded at P < 0.001 (Bonferroni corrected). (a) Mesial view of the left hemisphere. (b) Lateral view of the left hemisphere. (c) Occipital view of the left hemisphere. (d) Inflated gray-white matter boundary, occipital view. Positions of the sulci are indicated in dark gray.

Figure 11.

fMRI-constrained source analysis. (a) Significant fMRI activation (P < 0.001, Bonferroni correction) in condition II-AMlower used for dipole seeding and seeded dipolar sources. From top to bottom: (top) Coronal section through primary visual areas with dipoles V1/V2+ (cyan), V2/V3+ (red), V3/V3A+ (blue) display oriented in radiological convention (“R” indicates right). (middle) Transversal section through both hMT/V5+ sources: hMT/V5+ ipsilateral (magenta) and hMT/V5+ contralateral (green), oriented in radiological convention. The dipole symbol for hMT/V5+ ipsilateral (magenta) covers the small activation cluster for ipsilateral hMT/V5+. (bottom) 3-D view of cross-section through the head oriented in natural coordinates (“R” indicates right). (b) View of dipole positions and orientations in glass head model. All plots are oriented in radiological convention. Dipoles have the same colors as in (a). (c) Dipole source waves obtained by projecting the SSDW for condition II-AMlower onto the dipole model. Colors correspond to the respective dipole sources in (a, b). Dark center line indicates average signal; shaded areas indicate the 95% confidence interval obtained via bootstrap stastistics over 11 subjects. Dashed lines indicate the onset of activation in contralateral hMT/V5+ (at 88 ms), in V1/V2+ (100 ms), and the onset of consistent activity in the source assigned to V3/V3A+ (115 ms). Several sources (hMT/V5+ ipsilateral, V2/V3+, V3/V3A+) exhibit significant but very tiny activations before 60 ms. As stimulus processing in V1 was first observed at the scalp level around 60 ms we attribute these events to the increased noise of the SSDW signal.