Computer vision-guided open-source active commutator for neural imaging in freely behaving animals

Abstract

Significance: Recently developed miniaturized neural recording devices that can monitor and perturb neural activity in freely behaving animals have significantly expanded our knowledge of neural underpinning of complex behaviors. Most miniaturized neural interfaces require a wired connection for external power and data acquisition systems. The wires are required to be commutated through a slip ring to accommodate for twisting of the wire or tether and alleviate torsional stresses. The increased trend toward long-term continuous neural recordings has spurred efforts to realize active commutators that can sense the torsional stress and actively rotate the slip ring to alleviate torsional stresses. Current solutions however require the addition of sensing modules.

Aim: Here, we report on an active translating commutator that uses computer vision (CV) algorithms on behavioral imaging videos captured during the experiment to track the animal's position and heading direction in real time and uses this information to control the translation and rotation of a slip ring commutator to accommodate for accumulated mouse heading orientation changes and position.

Approach: The CV-guided active commutator has been extensively tested in three separate behavioral contexts.

Results: We show reliable cortex-wide imaging in a mouse in an open field with a miniaturized wide-field cortical imaging device. Active commutation resulted in no changes to measured neurophysiological signals.

Conclusion: The active commutator is fully open source, can be assembled using readily available off-the-shelf components, and is compatible with a wide variety of miniaturized neurophotonic and neurophysiology devices.

Keywords: calcium imaging; computer vision; miniaturized neurophotonics.

Fig. 1

Open-source CV-guided active, translating commutator system. (a) Principle of operation—mouse behavior video captured from an overhead camera is used to estimate the real-time position of the mouse location and heading direction, which is used to actively translate and rotate a slip ring commutator. (b) CAD schematic of the CV-guided active translating commutator system. (c) Photo of the CV-guided active translating commutator system. (d) Detailed CAD schematic of the motorized commutator module. (e) Photograph of the motorized commutator module.

Fig. 2

Evaluation of pose estimation toolboxes and convolution neural networks for real-time position and heading direction estimates. (a) Trajectory plot of a mouse performing the active place avoidance task (top) and polar histogram showing the distribution of the head direction ($n=3$ mice) in the same arena (bottom). The pose tracking was done using the DLC toolbox. (b) Trajectory plot of a mouse performing the Barnes maze task (top) and polar histogram showing the distribution of the head direction ($n=3$ mice) in the same arena (bottom). (c) Trajectory plot of a mouse in an open field arena (top) and polar histogram showing the distribution of the heading direction ($n=3$ mice) in the same arena (bottom). The pose tracking was done using the DeepLabCut toolbox. (d) Accuracy of real-time estimation of head position (top) and tailbase position (bottom) of the mouse during APA behavior for each of the three models: DLC implemented on MobileNetV2, DLC implemented on ResNet, and SLEAP implemented on U-Net. (e) Accuracy of real-time estimation of the head position (top) and tailbase position (bottom) of the mouse during Barnes maze behavior for each of the three models. (f) Accuracy of real-time estimation of the head position (top) and tailbase position (bottom) of the mouse during open field behavior for each of the three models. (g) Distribution of position inference time during APA behavior for the three models evaluated. (h) Distribution of position inference time during Barnes maze behavior for the three models evaluated. (i) Distribution of position inference time during open field behavior for the three models evaluated.

Fig. 3

CV-guided active translation and commutation. (a) Still image captured from the overhead video camera of the mouse in a 1.2-m-long linear track arena. (b) Plot of estimated mouse position in linear track as shown in panel (a) and the position of the commutator. Left: positions over the whole trial. Right: highlighting time duration indicated in a dashed rectangle in the plot on the left. (c) Plot of estimated mouse heading direction in the linear track arena as shown in panel (a) and the angular position of the commutator. Left: angular positions over the whole trial. Right: highlighting time duration indicated in a dashed rectangle in the plot on the left. (d) Still image captured from the overhead video camera of a mouse in an active place avoidance arena. (e) Plot of estimated mouse heading direction in the active place avoidance arena as shown in panel (d) and the angular position of the commutator. Left: angular positions over the whole trial. Right: highlighting time duration indicated in a dashed rectangle in the plot on the left. (f) Plot of the head position tracking accuracy of three DLC models implemented on MobileNetV2, trained on 40, 180, and 360 labeled frames. Left: head position tracking accuracy for the first 45 s of a Barnes maze trial. Right: highlighting time duration indicated in a dashed rectangle in the plot on the left. (g) Video frames from positions 1 and 2 as indicated in panel (f) for DLC model 1 with the head and tailbase tracking points marked on the images. (h) Plot showing the percentage of tracked frames below the 90% accuracy threshold for each of the three models over a whole trial (Video 1, mp4, 2.37 MB [URL: https://doi.org/10.1117/1.NPh.11.3.034312.s1]; Video 2, mp4, 9.42 MB [URL: https://doi.org/10.1117/1.NPh.11.3.034312.s2]; Video 3, mp4, 8.10 MB [URL: https://doi.org/10.1117/1.NPh.11.3.034312.s3]; Video 4, mp4, 5.21 MB [URL: https://doi.org/10.1117/1.NPh.11.3.034312.s4]).

Fig. 4

Wide-field calcium imaging in freely behaving mice during active commutation. (a) Pseudo-color DF/F $z$-score heat maps showing calcium activity progression during an active place avoidance trial during active commutation. (b) Top: mouse body angle tracking during a trial in the active place avoidance task. Gray lines denote active commutation periods to account for mouse angle changes greater than 90 deg, highlighted in green. Bottom: average DF/F $z$-score maps plotted for a wide range of regions of interest following the Allen Brain Atlas across one hemisphere of the brain. Gray lines denote active commutation periods to account for mouse angle changes $>90\mathrm{deg}$. (c) Top: peri-event time histograms for the average of 10 clockwise rotations of 720 deg with the active commutator. Bottom: peri-event time histograms for the corresponding average DF/F $z$-score across five regions of interest in the 10 clockwise rotations. Solid color lines indicate the average DF/F $z$-score for each region of interest. The gray solid line indicates the average of 1000 randomized bootstraps of the DF/F $z$-score data for the entire trial taken during the commutation period for each region of interest. The gray dashed line indicates the standard deviation of 1000 randomized bootstraps of the DF/F $z$-score data for the entire trial taken during the commutation period for each region of interest. (d) Top: peri-event time histograms for the average of 10 counterclockwise rotations of 720 deg with the active commutator. Bottom: peri-event time histograms for the corresponding average DF/F z-score across five regions of interest in the 10 counterclockwise rotations. Solid color lines indicate the average DF/F $z$-score for each region of interest. The gray solid line indicates the average of 1000 randomized bootstraps of the DF/F $z$-score data for the entire trial taken during the commutation period for each region of interest. The gray dashed line indicates the standard deviation of 1000 randomized bootstraps of the DF/F $z$-score data for the entire trial taken during the commutation period for each region of interest.