Learning of temporal motor patterns: an analysis of continuous versus reset timing

Abstract

Our ability to generate well-timed sequences of movements is critical to an array of behaviors, including the ability to play a musical instrument or a video game. Here we address two questions relating to timing with the goal of better understanding the neural mechanisms underlying temporal processing. First, how does accuracy and variance change over the course of learning of complex spatiotemporal patterns? Second, is the timing of sequential responses most consistent with starting and stopping an internal timer at each interval or with continuous timing? To address these questions we used a psychophysical task in which subjects learned to reproduce a sequence of finger taps in the correct order and at the correct times - much like playing a melody at the piano. This task allowed us to calculate the variance of the responses at different time points using data from the same trials. Our results show that while "standard" Weber's law is clearly violated, variance does increase as a function of time squared, as expected according to the generalized form of Weber's law - which separates the source of variance into time-dependent and time-independent components. Over the course of learning, both the time-independent variance and the coefficient of the time-dependent term decrease. Our analyses also suggest that timing of sequential events does not rely on the resetting of an internal timer at each event. We describe and interpret our results in the context of computer simulations that capture some of our psychophysical findings. Specifically, we show that continuous timing, as opposed to "reset" timing, is consistent with "population clock" models in which timing emerges from the internal dynamics of recurrent neural networks.

Keywords: computational modeling; human psychophysics; neural dynamics; recurrent networks; temporal processing; time estimation and production; timing.

Figure 1

Spatiotemporal Pattern Reproduction task data. (A) Top panel: Distribution of response times for each key press (dotted lines), with Gaussian fits (solid lines), for a complex pattern from one subject; Bottom panel: the target pattern. Colors cyan, blue, red, and green, correspond to fingers 2 (index) through 5 (pinky). A systematic anticipation is noticeable at the longer responses in this subject. Although some subjects tended to produce early responses and others late, across subjects there was a tendency to anticipate the target time, particularly at the last responses in the pattern. On the last day of training the difference between the response time and target time at the first element in the pattern was (mean ± SD) 25 ± 75 ms (p = 0.26), whereas for the last element it was −230 ± 320 ms (p = 0.03). The spatial error rate (trials with incorrect finger presses) decreased rapidly over the first few blocks: in the first block, on average, 33% of the trials had a spatial error; this fell to 8% in the second block, and 0.9% by the last block (12). (B) Overlaid normalized Gaussian fits from (A). Each distribution has the x axis divided by its mean and their y values divided by their peak value. Dotted lines correspond to the second response of a given finger. (C) Weber fraction (=coefficient of variation) as a function of mean response time. Same subject, all 3 days. Vertical dotted lines indicate the target times. (D) Variance as a function of mean response time squared (dashed lines), and linear fits according to the generalized Weber’s law (Eq. 1, solid lines).

Figure 2

Learning in the spatiotemporal pattern reproduction task. (A) Learning curves. Performance of all subjects over 3 days of training. RMSE: root-mean-square error between response times and target times, averaged in blocks of 20 trials. Statistically significant learners are in black, non-learners in gray. Some subjects classified as non-learners had very low RMSE values, but did not exhibit any significant improvement in performance from the first to last block, and were thus classified as “non-learners” (see Materials and Methods). (B) Mean slope k and mean intercept ${\sigma }_{\text{indep}}^2$ across the 3-days of training. Error bars represent the SEM. There was a significant decrease in both k and ${\sigma }_{\text{indep}}^2$ across training days (p = 0.013 and 0.011, respectively).

Figure 3

Reset versus continuous timing. Timing of a temporal pattern (thick black line) could be achieved by timing every interval t_n in isolation and resetting the timer at each response (Reset timing), or by timing continuously in absolute time T since the beginning of the pattern (Continuous timing). The predicted variance at every point in the sequence is different, since in Reset timing the variance at each new response adds to the previous variance, whereas in Continuous timing the variance is a function of absolute time squared (note that T₃ = t₁ + t₂ + t₃).

Figure 4

The psychophysical data is best fit by continuous timing. (A) Example of fits for Reset and Continuous timing from one subject: variance as a function of mean response time squared. Left column: periodic pattern; right column: complex pattern. Top row: training day 1; bottom row: training day 5. (B) Goodness of fit for Reset and Continuous: average R² after Fisher transformation with SE bars. (**) p < 0.0001, (*) p = 0.009.

Figure 5

Variance is smaller for periodic patterns. Mean variance at each response for the first and last days of training. Although both periodic and complex patterns had the same total duration (3 s), the variance of the response times at the last response is smaller for the periodic pattern.

Figure 6

Population clock model based on recurrent network dynamics. (A) Network architecture. The four output units are meant to represent the four fingers. Only the readout connections (depicted in red) are subject to training. (B) Reproduction of two complex spatiotemporal patterns (each with three different runs overlaid) after adjusting the weights of the recurrent units onto the readout units. Output traces are shifted vertically for visual clarity. Each pattern is triggered by a brief activation (100 ms) of the corresponding input unit at t = 0 (black thick trace, Pattern 1; gray thick trace, Pattern 2). The dashed black trace represents a (third) constant input to the recurrent network. Colored rasters represent a subset (20) of the recurrent units. In these units activity ranges from −1 (blue) to 1 (red).