Whole-body physics simulation of fruit fly locomotion

Abstract

The body of an animal influences how its nervous system generates behaviour¹. Accurately modelling the neural control of sensorimotor behaviour requires an anatomically detailed biomechanical representation of the body. Here we introduce a whole-body model of the fruit fly Drosophila melanogaster in a physics simulator². Designed as a general-purpose framework, our model enables the simulation of diverse fly behaviours, including both terrestrial and aerial locomotion. We validate its versatility by replicating realistic walking and flight behaviours. To support these behaviours, we develop phenomenological models for fluid and adhesion forces. Using data-driven, end-to-end reinforcement learning^3,4, we train neural network controllers capable of generating naturalistic locomotion^5-7 along complex trajectories in response to high-level steering commands. Furthermore, we show the use of visual sensors and hierarchical motor control⁸, training a high-level controller to reuse a pretrained low-level flight controller to perform visually guided flight tasks. Our model serves as an open-source platform for studying the neural control of sensorimotor behaviour in an embodied context.

Fig. 1. Constructing the female fruit fly body model.

a, Compilation of six datasets representing a single fly. Maximum intensity projections of confocal stacks showing head, thorax with abdomen, wings and legs. Scale bar, 1 mm. b, Left, a partial projection of the midleg confocal volume with the joints between the femur, tibia and tarsal segments indicated. Middle, a 3D mesh extracted from the volume. Right, a low-polygon leg model. Scale bar, 0.2 mm. c, An exploded low-polygon fly model (around 20,000 faces) showing all body segments. Scale bar, 1 mm. d, The geometric fly model assembled in Blender. e, The complete physics fly model in MuJoCo simulator in the default rest pose. f, Fly model in a flight pose with retracted legs. g, DoFs. Translucent bottom view with light-blue arrows indicating hinge joint axes pointing in the direction of positive rotation. Groups of three hinge joints effectively form ball joints. Cube: 6-DoF free joint required for free CoM motion in the simulator and is not a part of fly’s internal DoFs. h,i, Side view (h) and bottom view (i) of the geometric primitive (geom) approximation of body segments used for efficient collision detection and physics simulation. Blue, collision detection geoms; purple, geoms that have associated adhesion actuators; orange, wing ellipsoid geoms for simulating flight with the advanced fluid force model. j, Visualization of actuator forces generated when the model fly hangs upside down. The adhesion actuators of the front-right, middle-left and hind-right legs are actively gripping the ceiling (orange); the labrum (mouth) adhesors are also active; other actuators are inactive (white). The arrows visualize net contact forces proportional and opposite to the applied adhesion forces. k, Exaggerated posture showing the coordinated activation of the abdominal abduction and tarsal flexion actuators. Abdominal joints and tarsal joints (yellow) are each coupled with a single tendon actuator that simultaneously actuates multiple DoFs.

Fig. 2. Flight imitation.

a, Overview of RL set-up. A single policy network is trained to imitate the CoM position and body orientation across a dataset of 216 trajectories of freely flying Drosophila (around 43 s in total). The flight controller consists of a trainable MLP and a WPG. The motor command is the sum of the MLP and WPG outputs. Top right, one period of the fixed baseline wing-beat pattern produced by the WPG. Grey stripe indicates wing downstroke. b, Top, wing coordinate system and wing angle definition. Bottom, body coordinate system and example model sensory inputs: the direction to the goal CoM position and the gravity direction. c, Fluid model forces exerted on the left wing, and the corresponding wing kinematics, during a stable horizontal flight at 30 cm s⁻¹. d, Filmstrip of the model flying straight at 30 cm s⁻¹ during one full wing-beat cycle. e, Wing kinematics during a saccade manoeuvre produced by the model and real fly. f, Wings produce body movements through a phenomenologically modelled fluid. The real (black) and model (coloured) fly body pose while traversing a test trajectory. Circles, heads; lines, tails. g, Median and percentiles of body angular velocity, heading and speed for real and model flies during test saccades. The trajectories are aligned to peak acceleration at t = 0. Roll and pitch angular velocities (ω_x and ω_y) are similarly important in model flies’ and real flies’ turns. A small divergence between model and real occurs after the saccade. Solid lines, medians; shading, 25th–75th percentiles. h, Percentiles of errors between the model and the corresponding real fly’s body CoM, and orientation for 56 test trajectories. i, Wing angles during steady (small body acceleration) and unsteady (large body acceleration) wing beats for model and real flies in the test set. Large body accelerations are achieved by similarly small alterations to the median wing-beat pattern.

Fig. 3. Walking imitation.

a, Overview of RL set-up. A single policy network is trained to imitate a dataset of 13,000 snippets (around 64 min in total) of freely walking real Drosophila. Full body movements are imitated, including tracking CoM position, body orientation and detailed leg movements. b, Percentiles of errors between the model and the corresponding real fly’s body CoM, and orientation for 3,200 test walking trajectories. c, Filmstrip of the model walking straight at 2 cm s⁻¹ during one full leg cycle, with 8-ms steps between frames. d, Gait diagrams of the fly model tracking synthetic fixed-speed straight-walking trajectories at four speeds. Black stripes indicate the swing phase of leg motion. For each speed, the average number of legs simultaneously in stance position (on the ground) is indicated. e, Number of legs simultaneously in stance position averaged over walking snippet versus average walking speed in snippet. Top, model tracking test set trajectories. Bottom, entire walking dataset. Inset, the distribution of average walking speeds per snippet in the dataset. f, Distributions of swing onset phases of all legs relative to the front left leg L1 in walking trajectories with a mean speed in the range [1.2, 1.7] cm s⁻¹. Blue, fly model tracking test set trajectories; red, entire walking dataset. Dashed lines indicate circular medians. g, Learnt turning strategy. Top, xy projection of leg-tip trajectories in egocentric reference frame for model walking straight (black), turning left (green) and turning right (red), at a constant speed (2 cm s⁻¹). Leg-tip trajectories are shifted horizontally for clarity. Bottom, difference between (egocentric) left and right leg-tip swing length, averaged over all legs, at various walking speeds.

Fig. 4. Hierarchical controller reuse for vision-guided flight: altitude control (bumps) and obstacle avoidance (trench) tasks.

a, Overview of RL set-up. The policy uses vision to carry out flight at a given target speed and height while avoiding collision with terrain. In the bumps task, the terrain is a sequence of sine-like bumps across the flight path. The fly model must constantly adjust the altitude to maintain a constant target height above the bumpy terrain. In the trench task, the terrain is a narrow sine-shaped trench, requiring the fly to manoeuvre left and right. The terrain, target speed and height are randomly changed at each training episode. As in the flight imitation task, the flight controller combines policy network and WPG. The policy network consists of a CNN to process visual input from eye cameras; a high-level ‘navigator’ controller network; and reuses a low-level flight controller pretrained with the flight imitation task in Fig. 2. The high-level controller and the CNN are trained end-to-end with RL. The weights of the pretrained low-level controller are kept unchanged. b, Translucent top view of the head of the fly model, showing the placement of MuJoCo eye cameras. c, Top, high-resolution eye camera view for fly in a. Bottom, corresponding downsampled greyscale frames used as visual input. d, Top, time-lapse of flight produced by trained bumps-task policy. Bottom, example visual input frames captured by eye cameras. e, Side view of representative fly model trajectories in the bumps task. f, Percentiles of height and speed errors for 1,000 test bumps episodes. g, Left, time-lapse of flight produced by trained trench-task policy. Right, example visual inputs. h, Top-down view of representative fly model trajectories (left) and their corresponding flight height (right) in the trench task. i, Percentiles of height and speed errors for 1,000 test trench episodes.

Extended Data Fig. 1. Constructing a 3D model of a female fruit fly from confocal data.

a, Hi-polygon (around 22.6 million faces) model of the fly reconstructed in Blender. b, Simplified low-polygon model (around 20,000 faces). c, 3D model of the left-side coxae based on the confocal data. Red bars represent hinge joints, spheres represent ball joints. Arrows indicate anterior. d, Dorsal, sagittal and frontal views of the rigged Blender model in the rest position, the elements of the armature called ‘bones’ shown as elongated octahedrons.

Extended Data Fig. 2. Leg DoF analysis.

Inverse kinematics fits of the fly model legs to real Drosophila poses during grooming behaviour (392 poses in total.) To separate the effect of DoFs from fly-to-fly size variability, the model legs were rescaled to match the leg lengths in each individual reference pose frame. Individual model legs were fitted separately by simultaneously matching five leg key points located at four leg joints and leg tip. a,b, Absolute fitting errors for each leg key point, in units of body length, are shown for (a) all legs in rest position, (b) leg pairs T1 and T3 in grooming positions. Horizontal bars and corresponding values are median errors for each key point. Median errors across all key points in each leg pair are also indicated.

Extended Data Fig. 3. Adhesion and contact forces during walking on hilly terrain.

a, Schematic diagram of leg-floor contact forces for the fly model standing on an inclined surface. The adhesion actuator injects force in the normal direction, which in response increases the normal component of the contact force. This creates a larger margin between the tangential contact force component which resists slipping and the slip threshold (the friction cone boundary). b, Time-lapse of the trained policy rollout of the RL task where the fly model learns to use the adhesion mechanism to overcome sine-like hills. All the following panels correspond to this policy rollout. c, Adhesion actuator forces generated by the fly’s claws during the policy rollout shown in b. In our model, the largest adhesion force per leg is one fly body weight. The fly body weight, mg = 0.96 dyn, is shown for comparison. The leg-floor contacts are shown for clarity. d, Contact force norm during the policy rollout. The fly weight is shown for comparison. e, The difference between the slip threshold force and the tangential component of the contact force. This is the “margin” available to resist slipping under external forces and propulsion generation. f, The difference between the actual tangential contact force and the largest tangential contact force that would have been available without adhesion. Positive means the contact would have slipped without adhesion. Negative means the contact would not have slipped without adhesion (that is, the contact is inside the friction cone already without adhesion).

Extended Data Fig. 4. Sensitivity analysis of the dimensionless fluid model coefficients.

a,b, For viscous drag (a) and Kutta lift (b) components. Accuracy of imitation learning is shown in blue violins (body CoM error, 5 training runs, same test trajectories as in Fig. 2) and trajectory failure rate in orange bars (percentage of trajectories that crashed) for simulations varying each coefficient by up to 10×. Default values of the two coefficients are shown in bold. See the Supplementary Information and Supplementary Table 20 for fluid model details.

Extended Data Fig. 5. Example sensory input.

Sensory inputs to the flight imitation policy in Fig. 2. Vestibular sensors (velocimeter, accelerometer, gyro, gravity direction), proprioception (joint angles and joint velocities), and steering commands (target displacement and orientation) are shown. For clarity, we only show wing joint angles and omit the future preview of the steering command. More details in the Supplementary Information and Supplementary Table 6. Most of the observables are Cartesian vectors and their xyz components are correspondingly colour-coded as RGB. All inputs are represented in the fly’s egocentric reference frame.

Extended Data Fig. 6. Alternative DoFs and actuators.

Performance of the modified fly model with all 102 DoFs enabled and position actuators replaced with torque actuators. a,b, Flight imitation task, same as in Fig. 2 (a) and walking imitation task, same as in Fig. 3 (b). Top, Middle: Percentiles of errors between the fly model and target fly CoM position and body orientation. Bottom: Learning curve comparison between the original (blue) and modified (red) fly model. Episode return (e.g., cumulative episode reward) vs MuJoCo control steps during training is shown. The training is slower for the modified fly model. For flight, the episode return at end of training is similar in both cases. For walking, an additional multiplicative reward term is required to keep the (now enabled) wing DoFs in folded position and it causes most of the discrepancy between the two learning curves. This reward term is only approximately satisfied, causing a reduction of the episode return by a factor of ~0.69 in the trained model.

Extended Data Fig. 7. Distributed RL training architecture.

Multiple replicas of actors in MuJoCo environments collect experiences and feed them to a single replay buffer. The DMPO learner samples experiences from the replay buffer, updates the policy and critic network weights, and sends the updated weights to the actors’ copies of the policy.

Extended Data Fig. 8. Two-dimensional key points used to track fly walking.

a, A top-view video frame of a walking fly with key points inferred with the APT. b, Key-point definition. The fly schematic is provided by https://scidraw.io/ and is available at https://doi.org/10.5281/zenodo.3926097.