A cerebellar model for predictive motor control tested in a brain-based device

Abstract

The cerebellum is known to be critical for accurate adaptive control and motor learning. We propose here a mechanism by which the cerebellum may replace reflex control with predictive control. This mechanism is embedded in a learning rule (the delayed eligibility trace rule) in which synapses onto a Purkinje cell or onto a cell in the deep cerebellar nuclei become eligible for plasticity only after a fixed delay from the onset of suprathreshold presynaptic activity. To investigate the proposal that the cerebellum is a general-purpose predictive controller guided by a delayed eligibility trace rule, a computer model based on the anatomy and dynamics of the cerebellum was constructed. It contained components simulating cerebellar cortex and deep cerebellar nuclei, and it received input from a middle temporal visual area and the inferior olive. The model was incorporated in a real-world brain-based device (BBD) built on a Segway robotic platform that learned to traverse curved paths. The BBD learned which visual motion cues predicted impending collisions and used this experience to avoid path boundaries. During learning, the BBD adapted its velocity and turning rate to successfully traverse various curved paths. By examining neuronal activity and synaptic changes during this behavior, we found that the cerebellar circuit selectively responded to motion cues in specific receptive fields of simulated middle temporal visual areas. The system described here prompts several hypotheses about the relationship between perception and motor control and may be useful in the development of general-purpose motor learning systems for machines.

Fig. 1.

Schematic of the regional and functional neuroanatomy of the BBD. Gray ellipses denote different neural areas, black ellipses denote sensory input areas, and white ellipses denote motor areas. Arrows denote synaptic projections from one area to another. Black arrows ending in open arrowheads denote excitatory connections, black lines ending in a circular endpoint denote inhibitory connections, and gray arrows with dotted lines ending in filled arrowheads denote plastic connections. Visual input from a camera on the BBD was projected to the MT. The simulated cerebellar region consisted of a PN, PC (PC-Turn and PC-Velo), DCN (DCN-Turn and DCN-Velo), and input from the IO (IO-Turn and IO-Velo), where “Velo” refers to velocity. Neuronal units in the IO were driven by the IR proximity detectors, which in turn drove motor neurons for turning (Motor-Turn) and braking (Motor-Velo). Motor neurons were also driven by DCN (see Supporting Materials and Methods for more details).

Fig. 2.

The Segway BBD and its environment. (A) The BBD is built on the Segway RMP. The device navigated a path dictated by the orange traffic cones, which were spaced apart by a few inches. (B) The diagram shows the layout of the different courses. The lane dictated by the cones was 5 feet wide and ≈25 feet long. The device itself was ≈2 feet in diameter.

Fig. 3.

Training and testing the BBD on the middle course at a constant velocity with different delays in the eligibility trace learning rule. The motor error, which is the average IR signal over each lap obtained from five subjects, is shown in the plots. (A) Learning curves during training. Four different delay conditions, consisting of delays of 0, 2, 4, and 8 s, were used during training and testing. A control condition (no learning), in which the DCN→Motor connections were lesioned, was also tested. (B) During testing, the IR sensors were turned off, and the RMP relied on visual input alone. Trace delays of 2 and 4 s had significantly less motor errors for the five subjects than the no-learning condition (∗, P < 0.0005, Wilcoxon rank-sum test).

Fig. 4.

Training and testing the device on different curved courses. The mean motor error of five subjects is shown in the plots. (A) Learning curves during training on the gradual turn, sharp turn, and middle turn courses. (B–D) Motor errors were significantly lower in the test group, which had access to only visual cues, than in the control or “no learning” groups on the gradual course (B), middle course (C), and sharp course (D). ∗, P < 0.01, Wilcoxon rank-sum test.

Fig. 5.

Neuronal unit responses to different movements of neuronal units in the MT. Contour plots show the borders of MT neuronal unit activity before turning. The borders were found by tracing back from a motor unit that was active during a movement to the MT (see Results for details on the backtrace). The contours for the gradual, middle, and sharp courses are shown in red, green, and blue, respectively. Each contour depicts the MT neuronal units whose activities were above the 90th percentile. (A) MT responses during turns to the left (377 turn responses). (B) MT responses during turns to the right (280 turn responses).

Fig. 6.

Synaptic weights from the PN to the PCs for velocity control (PC-Velo) after training on different courses. Each matrix represents a mapping of the synaptic weights from the 30 × 40 PN area to each individual PC-Velo neuronal unit (11 units total). Each matrix has a retinotopic map that corresponds to visual input. The gray level of each pixel within each matrix indicates the strength, where the maximum is white (corresponding to 0.60), and the minimum is black (corresponding to 0.0). Weights were initialized to 0.5 (medium gray).