Active vision in freely moving marmosets using head-mounted eye tracking

Abstract

Our understanding of how vision functions as primates actively navigate the real-world is remarkably sparse. As most data have been limited to chaired and typically head-restrained animals, the synergistic interactions of different motor actions/plans inherent to active sensing-e.g., eyes, head, posture, movement, etc.-on visual perception are largely unknown. To address this considerable gap in knowledge, we developed an innovative wireless head-mounted eye-tracking system that performs Chair-free Eye-Recording using Backpack mounted micROcontrollers (CEREBRO) for small mammals, such as marmoset monkeys. Because eye illumination and environment lighting change continuously in natural contexts, we developed a segmentation artificial neural network to perform robust pupil tracking in these conditions. Leveraging this innovative system to investigate active vision, we demonstrate that although freely moving marmosets exhibit frequent compensatory eye movements equivalent to other primates, including humans, the predictability of the visual behavior (gaze) is higher when animals are freely moving relative to when they are head-fixed. Moreover, despite increases in eye/head-motion during locomotion, gaze stabilization remains steady because of an increase in vestibularocular reflex gain during locomotion. These results demonstrate the efficient, dynamic visuo-motor mechanisms and related behaviors that enable stable, high-resolution foveal vision in primates as they explore the natural world.

Keywords: ethology; eye-tracking; gaze stabilization; marmosets; vision.

Fig. 1.

Head-mounted eye-tracker assembly. (A) 3D render depicting different parts of the head-piece. The Ti- alloy scaffolding is custom designed and fabricated using Direct-Laser-Sintering (DLS) (B) 3D printed backpack that holds the two PCBs and the battery. (C) The animals are fitted with two 1 cm headposts that act as anchors for holding the head-piece. (D) Custom PCB board which receives data from camera sensors, IMU and saves it to a locally mounted SD card. (E) A marmoset wearing the fully assembled system (CEREBRO). (F) View from the three different cameras [eye camera (pink), world camera (yellow), and the external arena camera (cyan)] during a freely moving session. (G) Animals wearing CEREBRO exhibited no impairment in their free-moving characteristics such as percent movement, distance covered, and speed.

Fig. 2.

Iris/Pupil detection using segmentation Artificial Neural Network. (A) Conventional thresholding approach for pupil detection fails in scenes with variable light intensities. (B) Architecture of the segmentation neural network to crop out the iris. (C) The performance of the ANN matches closely to that of human annotation with relatively low training epochs (10 in this example) and relatively small amount of training data (~250 samples). (D) For calibrating the eye and world camera, the animals are presented with marmoset faces on a 120 Hz Liquid Crystal Display (LCD) monitor in a predetermined layout. (E) The calibrated eye movements can be overlaid on the world scene to determine the visual scene falling on the retina of the animal. (F) An illustration of the error estimation algorithm. (G) Accuracy comparison between Arrington eye tracker system and CEREBRO.

Fig. 3.

Eye/gaze accuracy in various experimental setups using CEREBRO. (A) Schematic for the experimental setup used to test accuracy of eye calibration in unrestrained animal. The setup consists of a dark box with a perch. The front of the box is covered with an IR longapss filter that does not allow visible light to pass except for a small window shown with a red dashed line in the Inset image. A galvanometric LASER system draws different geometric shapes on a screen for the animal to observe. (B) Schematic illustrating error estimation algorithm for freely moving eye calibration. (C) Maximum intensity projection of stimulus and eye in world position (red markers) for saccades and smooth pursuit trials in freely moving and head-restrained setup. (D) Accuracy comparison for eye calibration error for head restrained and freely moving animals during saccade trials and (E) smooth pursuit trails.

Fig. 4.

Eye tracking coupled electrophysiology in freely moving marmosets. (A) To validate use of CEREBRO with electrophysiology, subjects were presented with orientation grating and receptive field stimuli in a classical head fixed preparation. (B) The receptive field (Top) and orientation tuning curve (Bottom) is shown for three representative neurons recorded in marmoset V1 while using CEREBRO. (C) Example raster of 30+ single neurons in marmoset V1 while subjects are head-restrained (Left) and freely moving (Right) while wearing CEREBRO. The head (blue) and body (purple) speed of the marmoset using CEREBRO is plotted below the freely moving raster. (D) Example spike waveforms from five exemplar neurons demonstrate that we were able to stably record from same neurons in head-restraint vs. freely moving conditions.

Fig. 5.

Characteristics of eye/gaze behavior in various experimental setups using CEREBRO. (A) Heatmap of eye position in a restrained marmoset with dashed lines marking 2 SD mark. (B and C) distribution of horizontal and vertical eye position split by fixation (purple) vs. saccade (green) with corresponding dashed lines representing 2 SD of data variation. (D) Heatmap of eye position in an unrestrained marmoset. (E and F) Distribution of horizontal and vertical eye position split by gaze fixation (purple) and gaze shifts (green). Color-matched dashed lines represent 2 SD of variation in data.

Fig. 6.

Gaze characteristics in freely moving marmosets. (A) Representative session of activity pattern for marmosets in an open arena (200 cm × 100 cm × 240 cm). Gray dotted lines indicate periods of locomotion, while heat maps plot stationary occupancy. (B) Plots the estimate of the animal’s gaze (blue) with CEREBRO. Horizontal eye movement (red) and head yaw (green) are also plotted. Inset magnifies a 2 s period of time. (C) Horizontal eye movement and head yaw in freely moving marmosets are shown. Green dots indicate gaze fixations, while purple dots indicate gaze shifts. (D–H) Eye movements (red), head movements (green), and gaze shifts (red) in freely moving marmosets. (D) Plots the position (Top) and velocity (Bottom) of eye movements, head movements and gaze shifts <10° (Left), and >10° (Right). (E) Shows the maximum speed and gaze amplitude. Plots the latency to max speed (F), duration (G), and amplitude (H) as a function of gaze amplitude.

Fig. 7.

Characteristics of horizontal eye movements in freely moving marmosets. (A) An illustration of the gaze shifts in a freely behaving marmoset in two different states of locomotion (Left) and stationary (Right). (B) Animals exhibited noticeably different eye behavior in various contexts i.e. chaired, locomotion, and stationary. (C) Horz. eye movements have significantly less “approximate entropy” in head-fixed context as compared to freely moving. Although horz. gaze has the least approximate entropy of all the contexts. This shows that a freely moving gaze is computationally more regular than a head-fixed context. (D) Freely moving animal exhibits larger range of horz. eye amplitude and the speed of horz. eye movements is higher during locomotion than stationary state. Marmosets make more gaze shifts (E) during locomotion with higher speed (F) and larger amplitudes (G). (H) rms stabilization of head (yaw) and horz. gaze shows that despite poorer stability because of head movements, gaze remains remarkably stable during locomotion and stationary epochs.