Tiny visual latencies can profoundly impair implicit sensorimotor learning

Abstract

Short sub-100 ms visual feedback latencies are common in many types of human-computer interactions yet are known to markedly reduce performance in a wide variety of motor tasks from simple pointing to operating surgical robotics. It remains unclear, however, whether these latencies impair not only skilled motor performance but also the implicit sensorimotor learning that underlies its acquisition. Inspired by neurophysiological findings showing that cerebellar LTD and cortical LTP would both be disrupted by sub-100 ms latencies, we hypothesized that implicit sensorimotor learning may be particularly sensitive to these short latencies. Remarkably, we find that improving latency by just 60 ms, from 85 to 25 ms in continuous-feedback experiments, increases implicit learning by 50% and proportionally decreases explicit learning. This resulted in a dramatic reorganization of sensorimotor memory from a 45/55 to a 70/30 implicit/explicit ratio. This 70/30 ratio is more than double that observed in any previous study examining the effect of latency on sensorimotor learning, including a recent study which provided time-advanced visual feedback, suggesting that the low-latency continuous visual feedback we provided is critical for efficiently driving implicit learning. We go on to show that implicit sensorimotor learning is considerably more sensitive to latencies in the sub-100 ms range than to higher latencies, in line with the latency-specific neural plasticity that has been observed. This suggests a clear benefit for latency reduction in computer-based training that involves implicit sensorimotor learning and that across-study differences in computer-based experiments that have examined implicit sensorimotor learning might be explained by differences in unmeasured feedback latencies.

Fig. 1

Visual latency in a reaching task. (a) Participants made point-to-point reaching movements on a digitizing tablet while a cursor provided visual feedback on a horizontal screen positioned above the hand. Any latency in the visual display will make the cursor lag behind true hand motion. (b) Visuomotor rotation (VMR) with aiming paradigm. Users indicated their aim strategy before each movement by positioning an on-screen marker (green), allowing us to dissect total learning into explicit strategy vs. implicit components (the angle between the aiming marker and target in red vs. the angle between the hand motion and aiming marker in blue). (c)The overall visual latency in our experiments consisted of a base system latency that was optimized down to a 25 ms value that combined with experimentally-imposed delays of 0, 60 or 275 ms to yield latencies of 25, 85 or 300 ms. (d) Comparison the hand-cursor discrepancy induced by a 25 vs. 85 vs. 300 ms latency is in terms of velocity in target direction (left column) and distance traveled in target direction (right column). The black curves indicate the average hand velocity and distance profiles for each latency condition. The colored curves indicate the corresponding velocity and distance profiles for cursor motion. Note that because movement durations were rapid with durations of about only 300 ms, the shifts produced by these latencies led to large changes in the velocity and position profiles between cursor motion and hand motion.

Fig. 2

Small reductions in latency improve implicit and decrease explicit learning. (a–c) Learning curves for (a) overall, (b) implicit, and (c) explicit adaptation for the three latency conditions studied shows increasing implicit and decreasing explicit learning as latency is reduced, with overall learning largely unaffected by latency. The gray rectangle indicates the late learning period analyzed in panel d. Vertical dashed lines indicate trials following 60s breaks. These trials were excluded from the main analysis and hence not shown here, but are examined in Figure S3 in the supplementary materials. (d) Late learning vs. latency. While overall learning is largely unaffected by latency, implicit learning is increased by 50% when latency is reduced from 85 to 25 ms and doubled when latency is reduced from 300 to 25 ms. In contrast, explicit learning is reduced by 40% when latency is reduced from 85 to 25 ms and by 50% from 300 to 25 ms. (e) Sensitivity of implicit learning to changes in latency. This sensitivity, the rate at which implicit learning increases per unit of latency decrease, is nominally 5-fold higher across the sub-100 ms latency interval between 25 and 85 ms compared to the interval between 85 and 300 ms. Shading and error bars indicate ± SEM; note that error bars for overall learning in panel (d) are shown in white for visibility, as they would be occluded by the circle symbols otherwise.

Fig. 3

Extended learning data. (a–d) Same as Fig. 2a–d but for the 60-trial extended learning period. (e) Sensitivity of late implicit learning to changes in latency. The left pair of bars compares sensitivities across the 25–85 ms interval vs. 85–300 ms intervals for the extended learning data. The right pair shows this comparison for the combined learning and extended learning data. In both cases, the sub-100 ms interval displays markedly higher sensitivity of implicit learning to changes in latency than the longer latency interval. As in Fig. 2, post-break trials are not shown here, but are examined in Figure S3 in the supplementary materials. Shading and error bars indicate ± SEM.

Fig. 4

Latency reductions improve locally-generalizing implicit learning and decrease globally-generalizing explicit learning. (a) Generalization of VMR learning was measured with no-feedback movements across 19 different test directions that were centered on the trained target direction in 15° steps. (b–d) Effects of different latencies on the shape of (b) overall, (c) implicit, and (d) explicit generalization patterns. Thick lines indicate Gaussian fits (Eq. 1) used to dissect adaptation into its locally-generalizing and globally-generalizing components. Implicit learning primarily generalizes locally and is stronger at lower latencies, whereas explicit learning primarily generalizes globally and is stronger at high latencies. (e–g) Local and global generalization components extracted using Eq. 1. Decreased latencies display higher locally-generalizing and lower globally-generalizing overall learning, driven by increases in the primarily locally-generalizing implicit learning and decreases in the primarily globally-generalizing explicit learning. Error bars indicate ± SEM.