Sensorimotor delays constrain robust locomotion in a 3D kinematic model of fly walking

Abstract

Walking animals must maintain stability in the presence of external perturbations, despite significant temporal delays in neural signaling and muscle actuation. Here, we develop a 3D kinematic model with a layered control architecture to investigate how sensorimotor delays constrain the robustness of walking behavior in the fruit fly, Drosophila. Motivated by the anatomical architecture of insect locomotor control circuits, our model consists of three component layers: a neural network that generates realistic 3D joint kinematics for each leg, an optimal controller that executes the joint kinematics while accounting for delays, and an inter-leg coordinator. The model generates realistic simulated walking that resembles real fly walking kinematics and sustains walking even when subjected to unexpected perturbations, generalizing beyond its training data. However, we found that the model's robustness to perturbations deteriorates when sensorimotor delay parameters exceed the physiological range. These results suggest that fly sensorimotor control circuits operate close to the temporal limit at which they can detect and respond to external perturbations. More broadly, we show how a modular, layered model architecture can be used to investigate physiological constraints on animal behavior.

Keywords: D. melanogaster; computational biology; generative model; locomotor control; neuroscience; quantifying behavior; sensorimotor delay; systems biology; walking.

Figure 1.. A layered model of fly walking that incorporates link-and-joint dynamics and sensorimotor delays.

(A) The neural systems that control walking are hierarchically organized. The brain sends high-level commands (e.g. walking speed and direction) to the ventral nerve cord (VNC). Approximately 70 motor neurons (MNs, Azevedo et al., 2024) control each leg. The cell bodies and dendrites of the MNs are in the VNC, and their axons innervate muscles in the leg. Each leg muscle is actuated, after some delay, following activation by one or more motor neurons. Sensorimotor circuits in the VNC also receive delayed sensory feedback via proprioceptors. (B) Multi-layered walking model architecture. Body-world interactions are mediated by a dynamics model. Proprioceptive feedback consists of joint angles and angular velocities (Mamiya et al., 2018). Each leg has an optimal controller operating at 600 Hz that interfaces with the dynamics model and a trajectory generator that generates realistic kinematics. The trajectory generator is learned from data and operates at 300 Hz. It interfaces with the phase coordinator, a Kuramoto oscillator that maintains inter-leg coupling. The trajectory generator and optimal controller mimic local circuits within each leg, so they do not interact with other legs. Phase coupling is the only information shared between all legs. (C) Dynamics model of a fly leg, derived from link-and-joint models and Euler-Lagrange equations. (D) Schematic of model training and execution process. Model parameters for each module are tuned independently, then the modules are assembled to generate walking with delays and in response to unexpected perturbations.

Figure 2.. Walking model simulations produce realistic joint kinematics.

(A) Example time series of femur-tibia flexion (R1) and femur rotation (L2) for three different walking speeds: 8, 10, and 12 mm/s. Real data (orange) exhibited more variability than simulations (blue). (B) Angle vs. computed per-leg phase of femur-tibia flexion on leg R1 and femur rotation on leg L2 for four different walking speeds. Each plot contains data from 4 walking bouts with different initial conditions. (C) Average differences between model simulations and data, over a range of forward walking, turning, and side-stepping speeds over 500 distinct bouts. The dotted line (5.56 degrees) indicates uncertainty associated with markerless 3D joint tracking (Karashchuk et al., 2021). All simulations used a sensory delay of 10 ms and a motor delay of 30 ms, based on values measured experimentally with electrophysiology from leg sensory and motor neurons/muscles (Tuthill and Wilson, 2016b; Azevedo et al., 2020).

Figure 3.. The model sustains robust walking during and following perturbations.

(A) Schematic of how perturbations are applied to the walking model. (B, C) Example time series of femur-tibia flexion on leg R1 and femur rotation on leg L2 for three different walking speeds (8, 10, and 12 mm/s) before, during, and after perturbation. In panel B, an impulse perturbation of size 3.75 rad/s is applied at a single instant (dotted line), and its effects are analyzed over a brief time window (shaded blue area). In panel C, stochastic perturbations of size 1.875 rad/s are applied over a time window (shaded green area). Angle trajectories are visibly different during the analysis windows, but recover afterward. Perturbation effects appear similar across speeds. All simulations used a sensory delay of 10 ms and a motor delay of 30 ms.

Figure 4.. The walking model sustains realistic walking kinematics following impulse and persistent stochastic perturbations.

(A) Method for computing kinematic similarity between real data and simulated walking. First, the full set of experimental data (from flies) is used to compute a Gaussian kernel density estimator (KDE). To quantify the similarity (to data) of a given bout of simulated walking, we apply the KDE to evaluate the log probability density function of each bout, a scalar value we refer to as kinematic similarity (KS). High KS indicates that the perturbed walking resembles the unperturbed walking from data, while low KS indicates that the perturbed walking deviates from data and results in unnatural angles (as seen at 40 ms motor delay). The average KS value of bouts from experimental data was –1.627. (B, C) Kinematic similarity of simulated walking during perturbations for impulse and persistent stochastic perturbations. For each square of the heatmap, four simulations with different initial conditions were simulated and averaged. Simulations became less similar to data with increased perturbation strength. All simulations used a sensory delay of 10 ms and a motor delay of 30 ms. Perturbation velocities were drawn from a random uniform distribution with mean perturbation strength (rad/s) and standard deviation of 0.1 × mean.

Figure 5.. The model maintains robust walking under persistent stochastic perturbations over specific ranges of motor and sensory delays.

(A, B) Example time series of femur-tibia flexion on leg R1 and femur rotation on leg L2 under various values of motor (10, 20, 30, 40 ms) and sensory delay (0, 5, 10, 15 ms). Perturbation effects became more noticeable with increasing delay values. (C, D, E) Similarity of during-perturbation walking to data across delay values, perturbation strengths, and forward speeds. For each square of the heatmap, four simulations with different initial conditions were simulated and evaluated. At low delay values, simulated walking maintained high similarity even under large perturbations. As perturbation strength and delays increased, simulated walking became less similar to data; the effect was more pronounced with increased delays. When we fixed one delay value and varied the other, the model maintained realistic walking (KS > – 1.6) up to about 30 ms of motor delay and 10 ms of sensory delay across a range of conditions. When we allowed both motor and sensory delay to vary, the model maintained realistic walking when the sum of the delays was no more than about 45 ms. (F) Post-perturbation walking with motor and sensory delays. The model was unable to recover from perturbations for large delay values. Unless otherwise stated, forward speed = 12 mm/s, and perturbation strength = 1.875 rad/s.

Figure 6.. Detailed schematic of optimal controller.

The controller receives delayed sensory information from the dynamics model, as well as future trajectory information from the trajectory generator. The controller interacts with the dynamics model via a delayed motor action signal. The internal structure of the controller is that of a standard output feedback controller, which incorporates estimation via a Kalman filter. Dotted lines indicate muscle and sensor delays.

Appendix 1—figure 1.. Velocity and phase distributions in data justify the selection of model simulation parameters.

(A) Cumulative distribution functions and number of frames associated with various forward, rotation, and side-step velocities in data. Right/left (side-step) walking peaked at 0 mm/s, while forward/backward walking achieved a sustained peak from approximately 4 mm/s to 10 mm/s. Rotation velocity peaked at 0 rad/s. Most model-generated simulations in the paper adhered to this range. (B) Rotation vs. side-step velocity. Each dot on the scatter plot represents a single bout from data. Rotation velocity and side-step velocity were highly correlated with one another in data. (C) Phase offsets between legs in data over a range of walking velocities and rotational velocities. Side-step velocities were omitted from the study since (as previously demonstrated) they were highly correlated with rotation velocity. Phase offsets remained relatively constant over a range of forward and rotational velocities.

Appendix 2—figure 1.. Simulated walking reproduced realistic joint angles.

Angle vs. computed per-leg phase of all joints on all legs for four different walking speeds. All simulations used a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values. Some oddities were observed in real data, for instance in L3 femur flexion and R1 body-coxa flexion. The model does not quite capture these oddities.

Appendix 3—figure 1.. Simulated walking reproduced realistic joint angular velocities.

Angular velocity vs. computed per-leg phase of all joints on all legs for four different walking speeds. All simulations used a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values.

Appendix 4—figure 1.. Simulated walking resembled data over a range of forward, turning, and side-stepping velocities.

Average differences between model simulations over 500 distinct bouts. All simulations use a sensory delay of 10 ms and motor delay of 30 ms, consistent with experimental values. The dotted line (5.56 degrees) indicates uncertainty associated with data collection.

Appendix 5—figure 1.. Simulated walking reproduced across- and within-leg coupling characteristics of data.

(A) Phase coupling across legs. We compared phases of representative joints across legs. Model coupling qualitatively resembled data coupling, with moderate variations in phase difference (i.e. peak location) and synchronization strength (i.e. peak height). A strong peak in probability density indicates synchronization, whereas broader peaks indicate weak synchronization. A single peak at zero on the horizontal axis indicates that the two joint phases are coupled to match; a single peak elsewhere indicates that the two joint phases are coupled with some phase offset. Simulations were performed over a range of forward walking, turning, and side-stepping speeds over 500 distinct bouts. (B, C) Phase coupling within each leg. For each leg, we compared phases between a representative joint for the leg (denoted ‘target’ on the image) and other joints on the leg. Gaps are present as we did not include all five joints for all legs in the model. All synchronizations (i.e. peaks) in data are reproduced by the model. However, simulations generally exhibited stronger synchronization (i.e. higher peaks).

Appendix 7—figure 1.. Model generated robust walking under persistent stochastic perturbations over select ranges of motor and sensory delays, revealing fundamental constraints on delay values.

(A, B) Example simulated time series of femur-tibia flexion on leg R1 and femur rotation on leg L2 under various values of motor (10, 20, 30, 40 ms) and sensory delay (0, 5, 10, 15 ms). Perturbation effects were more noticeable with increased delay values. (C, D, E) KS of simulated during-perturbation walking to data, for various values of delay, perturbation strength, and forward speeds. For each square of the heatmap, four simulations with different initial conditions were simulated and evaluated. As perturbation strength and delays increased, simulated walking became less similar to data; the effect was more pronounced with increased delays. When we fixed one delay value and varied the other, the model maintained realistic walking (KS ${\displaystyle >-1.6}$) up to about 25 ms of motor delay and 8 ms of sensory delay across a range of conditions. When we allowed both motor and sensory delay to vary, the model maintained realistic walking when the sum of the delays was no more than about 40 ms. (F) Post-perturbation walking with motor and sensory delays. The model is able to recover from perturbations for very large values of delay. Unless otherwise stated, forward speed = 12 mm/s, and perturbation strength = 5 rad/s.

Appendix 8—figure 1.. Principal components model the range of kinematics observed in the data.

(A) An example time series of R1 femur-tibia flexion along with principal components of all the angles. The first two principal components (PC1 and PC2) capture the step cycle of walking. The later principal components capture increasingly higher-frequency aspects of the angle trajectories. (B) Explained variance of the principal components. Seven components explain over 81% of the variance and 12 components explain over 91% of the variance in joint angles.

Appendix 9—figure 1.. Results on sensory delay thresholds are generally robust to varying principal components analysis (PCA) dimension used in the kinematic similarity metric.

Effect of varying principal component number on kinematic similarity (KS) measurement for stochastic (A) and impulse (B) perturbations. The results look similar across components for impulse perturbations. For stochastic perturbations, the range of similar walking decreases as we increase the number of components used to evaluate walking kinematics. As higher principal components represent higher temporal frequencies, it is likely that higher frequencies are impacted at the edge of stability before walking collapses entirely at increasing delays.