Eye movements shape visual learning

Abstract

Most people easily learn to recognize new faces and places, and with more extensive practice they can become experts at visual tasks as complex as radiological diagnosis and action video games. Such perceptual plasticity has been thoroughly studied in the context of training paradigms that require constant fixation. In contrast, when observers learn under more natural conditions, they make frequent saccadic eye movements. Here we show that such eye movements can play an important role in visual learning. Observers performed a task in which they executed a saccade while discriminating the motion of a cued visual stimulus. Additional stimuli, presented simultaneously with the cued one, permitted an assessment of the perceptual integration of information across visual space. Consistent with previous results on perisaccadic remapping [M. Szinte, D. Jonikaitis, M. Rolfs, P. Cavanagh, H. Deubel, J. Neurophysiol. 116, 1592-1602 (2016)], most observers preferentially integrated information from locations representing the presaccadic and postsaccadic retinal positions of the cue. With extensive training on the saccade task, these observers gradually acquired the ability to perform similar motion integration without making eye movements. Importantly, the newly acquired pattern of spatial integration was determined by the metrics of the saccades made during training. These results suggest that oculomotor influences on visual processing, long thought to subserve the function of perceptual stability, also play a role in visual plasticity.

Keywords: eye movement; plasticity; remapping; visual learning.

Fig. 1.

Stimulus timing and experimental procedure. (A) Temporal sequence of visual stimuli in relation to onset of a saccadic eye movement (10). A fixation target (ft) appeared, and after fixation was acquired for 500 to 750 ms, four random dot kinematograms (RDKs) appeared at locations positioned symmetrically about the ft. Subsequently a saccade target (st) appeared along with a cue to indicate the location of the RDK (S1) to be discriminated. After a variable interval, the motion of S1 became coherent, and on most trials the motion of another RDK (S2) became coherent as well. The saccade was then executed, and the perceived motion direction was reported. A saccade could be horizontal (as shown here) or vertical. (B) Step-by-step evolution of visual stimulus presentation. The fixation period (1) was followed by the appearance of the st (2) and then a flashed green Gaussian blob to indicate the location of S1 (3). Trials with only S1 were called the Just-S1 condition (3a). In trials with two simultaneous motion signals (S1 + S2), depending on the relative locations of the cue and the saccade direction, the condition was called Remapping (3b), Diagonal-Control (3c), or Orthogonal-Control (3d). (4) After the 100 ms presentation of the motion signals, the RDKs became incoherent again for 400 ms. Observers were instructed to report the direction of S1 at the end of each trial. Note that, in this example, S1 is on the top right, but could occupy any of the other three locations on different trials.

Fig. 2.

Evolution of performance in the saccade task. (A) Motion sensitivity in the saccade condition for example observer 1. Each point on the x axis represents one group of 800 trials, which were subdivided into 200 trials of each of the four conditions listed in the Upper Right Inset. In total, about 4,000 trials were analyzed (SI Appendix, Supplementary Information Text). A mean d′ value (± SEM) for each of the five analysis periods was computed according to the observer’s performance at each coherence level, expressed as a mean of the fraction of correct trials within the period. In all five periods, the d′ values were significantly higher for the remapping condition (black) than for the Just-S1 condition (green). By comparison, performance in the other control conditions was not significantly different from performance in the Just-S1 condition. (B) Motion sensitivity in the saccade task for observer 2. This observer’s performance in the Remapping condition (black) was similar to that in the Just-S1 condition (green): Training significantly improved performance in each condition, but there was no advantage for remapping over any other condition (P > 0.05, permutation test). Asterisks: statistically significant differences between Remapping and Just-S1 conditions (P < 0.05, permutation test). Error bars show SEM.

Fig. 3.

Evolution of motion integration through training on saccade trials. (A) Comparison of the remapping effect in the first (green) and the last (blue) eight sessions for the individual observers who showed a remapping effect. Error bars indicate SEM. (B) Effects of S2 on perception of S1, combined across observers. To describe quantitatively the timewise influence of training in the saccade task, for each observer, trials were divided into 27 steps: the first step contained the data for the first four sessions (∼200 saccade trials), and the step was moved forward 30 trials at a time. Each point indicates the mean improvement observed with presentation of S2 in the Remapping (black), Diagonal-Control (blue), and Orthogonal-Control (orange) conditions, combined across observers. Solid lines show the best-fitting linear regressions, indicating no learning effect specific to the presentation of S2 (Remapping condition–Just-S1 condition: slope = 0.0016, P = 0.11, SE of the slope = 0.001). Dashed lines around the regression lines show the confidence interval for the regression estimate.

Fig. 4.

Evolution of performance in the fixation task. (A) Performance of the example observer who showed a remapping effect during saccades. Same conventions as in Fig. 2. In the Like-Remapping condition (black), the mean d′ values are initially similar to those of the Just-S1 (green) and control (blue, orange) conditions. A significant improvement for Like-Remapping relative to Just-S1 trials emerged in the fourth group of sessions (P < 0.05, permutation test). In the other control conditions, no significant increase of performance occurred in any period in comparison with the Just-S1 condition. (B) Performance of observer 2 during fixation trials. Although performance improved through time, no location-specific improvement was found for the Like-Remapping (black) or any other condition. Error bars show SEM. Asterisks: statistically significant differences between Remapping and Just-S1 conditions (P < 0.05, permutation test).

Fig. 5.

Prediction of visual learning with linear model based on number of trials and remapping. A statistical model attempted to capture the performance of observers on the Like-Remapping trials relative to the Just-S1 trials during fixation. Each dot corresponds to a single observer’s results at the end of training. (A) The solid line along with its confidence bounds (dashed lines) corresponds to a model that considered only the number of trials completed as a predictor ( = 0.32, BIC = −21.47). (B) A model that considered only the strength of remapping as a predictor ( = 0.52, BIC = −25.07). (C) A model that used both factors as predictors ( = 0.89, BIC = −38.98).

Fig. 6.

Evolution of motion integration through training on fixation trials. (A) Comparison of motion integration in Like-Remapping in the fixation task. The d′ differences were computed by subtracting the d′ values in the Just-S1 condition from the Like-Remapping condition during the first eight sessions (green) and last eight sessions (blue) for the individual observer who showed a remapping effect during the saccade task. (B) In the fixation task, d′ differences were computed for each observer by subtracting the d′ values in the Just-S1 condition from those obtained in the conditions with two motion signals (S1 + S2) for the last eight sessions. Differences larger than zero indicate an increase in motion sensitivity in the S1 + S2 conditions (motion integration). Motion integration of 25% (d′ difference = 0.25) was found in the Like-Remapping condition for observers with remapping during the saccade task. The d′ difference between Like-Remapping and Just-S1 conditions was significantly different from that between the other S1 + S2 control conditions and Just-S1 (P < 0.05, WSR test, FDR corrected). The dashed line indicates the actual remapping effect during the saccade trials (SI Appendix, Fig. S1B) (C). Effect of training during the fixation task. For each observer, trials were divided into 27 steps: the first step contained the data for the first four sessions (∼150 fixation trials), and the step was moved forward 25 trials at a time. Each point indicates the mean difference between the S1 + S2 and the Just-S1 conditions across all observers who showed a remapping effect during saccades. These points are fitted with linear regression (solid lines). The black dashed lines around the regression lines show the confidence interval for the regression estimate. The red dashed line indicates the remapping regression during the saccade task (Fig. 3). Error bars show SEM. Asterisks: statistical significance (P < 0.05).

Fig. 7.

Relationship between saccade latency and remapping. (A) Time course of motion integration in remapping. Motion sensitivity in the Remapping condition relative to that in the Just-S1 condition is pooled for all observers and plotted through time in 25-ms windows, according to the timing of motion signal offset relative to saccade onset. Negative time values indicate that motion signals in S1 and S2 appeared and ended before saccade onset. Filled and open triangles indicate, respectively, mean timing difference for observers with and without remapping during saccade trials. When the stimulus was presented longer than ∼175 ms before a saccade (vertical bold dashed line), no remapping was observed. Error bars show SEM. (B) Distribution of saccade latencies. Observers who showed a remapping effect during saccades (Left; median 235.66 ms) had shorter latencies than those who did not show remapping (Right; median 398.76 ms). (C) Remapping is linked to stimulus timing relative to saccade onset. We separated the saccade trials for all observers into those in which the time between the stimulus and the saccade was greater than and less than 175 ms. For observers who showed remapping in the full dataset, there was no increase in motion sensitivity in the Remapping conditions for the longer-latency saccades (mean d′ difference ± SEM = −0.09 ± 0.12), while for trials with shorter latencies we observed motion integration in the Remapping condition (mean d′ difference ± SEM = 0.28 ± 0.06). For observers who did not show remapping (right), no motion integration was observed in trials with long latencies (mean d′ difference ± SEM = −0.12 ± 0.11), but clear remapping was observed for the short latency trials (mean d′ difference ± SEM = 0.18 ± 0.065).