Advances in non-invasive tracking of wave-type electric fish in natural and laboratory settings

Abstract

Recent technological advances greatly improved the possibility to study freely behaving animals in natural conditions. However, many systems still rely on animal-mounted devices, which can already bias behavioral observations. Alternatively, animal behaviors can be detected and tracked in recordings of stationary sensors, e.g., video cameras. While these approaches circumvent the influence of animal-mounted devices, identification of individuals is much more challenging. We take advantage of the individual-specific electric fields electric fish generate by discharging their electric organ (EOD) to record and track their movement and communication behaviors without interfering with the animals themselves. EODs of complete groups of fish can be recorded with electrode arrays submerged in the water and then be tracked for individual fish. Here, we present an improved algorithm for tracking electric signals of wave-type electric fish. Our algorithm benefits from combining and refining previous approaches of tracking individual specific EOD frequencies and spatial electric field properties. In this process, the similarity of signal pairs in extended data windows determines their tracking order, making the algorithm more robust against detection losses and intersections. We quantify the performance of the algorithm and show its application for a data set recorded with an array of 64 electrodes distributed over a 12 m² section of a stream in the Llanos, Colombia, where we managed, for the first time, to track Apteronotus leptorhynchus over many days. These technological advances make electric fish a unique model system for a detailed analysis of social and communication behaviors, with strong implications for our research on sensory coding.

Keywords: animal biometric system; behavioral tracking; electric fish; remote sensing; tracking.

Figure 1

Recording systems, electrode arrangements, and corresponding signals of recorded electric fish. (A) Two of the Raspbery Pi-based 16-channel amplifiers and recorders used for an array with 32 electrodes. (B) Monopolar stainless-steel electrode on headstage used for recordings in the field and laboratory experiments (after Henninger, 2015). (C) Recording setup used to record a population of A. leptorhynchus in the Rio Rubiano, Colombia, in 2016. Sixty-four electrodes were mounted on PVC-tubes and arranged in an 8 × 8 grid covering an area of 3.5 × 3.5 m². (D) Snapshot of the electric signals recorded with the setup shown in (C). The top left panel corresponds to the most upstream electrode mounted on the tube closest to the river bank. (E) Recording setup used to record electric signals of pairs of A. leptorhynchus during competitions in a laboratory experiment (Raab et al., 2021). Fifteen electrodes were uniformly distributed at the bottom of the aquarium and one electrode was placed in the central tube the fish compete for. (F) Snapshot of electric signals recorded during the competition experiment shown in (E). The signal framed in gray is from the central electrode located in the optimal tube. The EOD waveform shows the characteristic shoulder that is generic for EODs of A. leptorhynchus.

Figure 2

EOD frequency extraction from recordings with an electrode array. As an example, a 3 min snippet of a recording with the 8 × 8 array from Rio Rubiano, Colombia, taken during the day of April 10^th, 2016 is shown. (A) Spectrograms from three different electrodes. Warmer colors represent increased power in respective frequencies. EOD frequencies of individual A. leptorhynchus remain rather stable, except during electrocommunication (e.g., EOD frequency trace starting at ~ 917 Hz). A non-logarithmic PSD extracted at time 50 s indicated by the dotted line is shown at the side of each panel. (B) The summed up spectrogram over all electrodes contains distinct traces from many different fish. (C) Peaks are detected in the summed up power spectra that are then clustered into frequency groups of a fundamental frequency and at least two of its harmonics, corresponding to a specific fish (Henninger et al., 2020). Fundamental EOD frequencies, their corresponding powers in each electrode and their detection times are stored for subsequent tracking.

Figure 3

Frequency and field errors. (A) Summed spectrogram of a 30 s long part of the recording shown in Figure 2B. For each electric fish signal, potential connection partners are limited by a time difference threshold, Δt_thresh = 10 s, and a frequency difference threshold, Δf_thresh = 2.5 Hz. For a given signal α with EOD frequency f_αi at time step i (dark blue dot), potential connection candidates β at different times j (light blue dots) need to be within these thresholds (box), whereas signals beyond these thresholds (black dots) are not considered. (B) Absolute frequency differences, Δf Equation (2), are mapped (red lines) to frequency errors, ε_f, using a logistic function, Equation (4) (line), favoring small frequency differences. (C) The field error as the second tracking parameter is based on spatial profiles, Equation (5), of signal powers over all electrodes (black dots). The field difference, ΔS, is computed as the Euclidean distance, Equation (6), between the spatial profiles, Equation (5), of potential signal pairs (columns). With decreasing similarity (columns left to right) the field difference increases. Displayed signal pairs (columns) were selected to illustrate the full range of possible field differences and are unrelated to (A). Spatial profiles were interpolated with a gaussian-kernel for illustrative purposes. (D) To obtain normalized field errors, ε_S, in a range similar to the one of the frequency errors, ε_f, each field difference is set into perspective to a representative cumulative distribution [Equation (7), black line] of field differences obtained by collecting all potential field differences of a manually selected 30 s window in the recording. The cumulative distribution of potential field differences is computed only once per recording for a 30 s window where fish are active (night time). This way we incorporate a broad distribution of possible field differences when determining field errors. The examples from (C) are marked by respectively colored dots.

Figure 4

Distance cube containing all distances, ε_{α, β} Equation (3), for possible signal pairs α and β within the current tracking window. Each layer, referring to a time step i, contains the distances between all signals α_i detected at this time and their potential signal partners β_j detected maximally 10 s after signal α_i (ΔI time-steps after i). Distances in gray layers correspond to signal pairs where one signal partner could potentially have a smaller distance to a signal outside the error cube. Only connections based on the distances in the central black layers can be assumed to be valid, since all potential connections of both signal partners are within the error cube. Connections established for the black layers are assigned to signal traces obtained in previous tracking steps in a second step.

Figure 5

Tracking within a data window. Signals detected in a 30 s data window are connected to each other and assigned to fish identities according to their distance ε, Equation (3). Signal pairs with smaller distances are connected first. With increasing distance values, more connections and identities are formed, complemented, or merged, ensuring no temporal overlap. Different stages of this tracking step are displayed in (A–C). (A) Twenty percent of all possible connections of the displayed tracking window are formed. At this tracking stage a multitude of separate signal traces (different colors) are still present. (B) Forty percent of all possible connections of the displayed tracking window are formed. (C) Final output of the tracking step. All possible connections of the displayed tacking window are formed. The remaining three EOD frequency traces (in the displayed time and frequency segment) correspond to three different fish identities. Only signal pairs within the central 10 s of an 30 s tracking window (vertical lines) are assigned to already established fish identities from previous tracking windows. The summed spectrogram of a 30 s long part of the recording shown in Figure 2B is shown in the background.

Figure 6

Assembly of tracking results over data windows. (A) New fish identities established within the current tracking window (gray and black bar on top). Only the central 10 s of these EOD frequency traces (solid traces; black bar) can be assumed to be valid since signals before and after (transparent traces; gray bars) have potential signal partners outside the tracking window. (B) Additional display of EOD frequency traces established in previous iterations of the tracking algorithm. (C) Signal traces are connected according to the smallest possible distance measure between any signal between the last 10 s of the established fish identities (10 s < t < 10 s) and the central 10 s of the new fish identities (10 s < t < 20 s). In the example shown, the distance between the origin signal (black dot) and the target signal (green dot) is the smallest between these two signal traces, accordingly the two signal traces are merged (green and orange lines). An alternative signal (red dot) has a larger distance to the origin signal. (D) Final result of the tracking algorithm that will be used for the next iteration.

Figure 7

Graphical user interface for validating and fixing tracking results. The user is presented with the tracked signal traces (EOD frequency traces) displayed on top of a spectrogram summed up across recording electrodes. The user can delete, cut, and connect signal traces or delete signals not originating from electric fish.

Figure 8

Performance of the tracking algorithm. Conflicts appear if signals could be connected to multiple different fish identities, that have been manually corrected and checked post-hoc (Figure 7). In most but not all cases, correct connections have smaller signal differences or errors (blue) than wrong connections (red). Shown are kernel density estimates (KDE) for the various signal differences, errors, and distances. The overlap of the distributions was quantified by the AUC of an ROC-analysis as indicated in the right column. (A) EOD frequency differences, Δf Equation (2). A logistic function, Equation (4) (black line), translates EOD frequency differences to frequency errors, ε_f. (B) Field differences, ΔS, Equation (6). The cumulative distribution (black line) of field differences of all pairings, not only from conflicts, translates field differences to field errors, ε_S, Equation (7). (C) Frequency error, ε_f, Equation (4). (D) field error, ε_S, Equation (7). (E) Combined distance measure, ε, Equation (3). Note, that frequency and field errors (C,D) are mapped via monotonically increasing functions from signal differences (A,B) and thus result in the same fraction of correct connections and AUC values. However, the distance measure combining both field and frequency error performs best.

Figure 9

Long-term field recording of A. macrostomus, a member of the A. leptorhynchus species group, in Colombia, 2016. EODs were recorded with a 64 channel electrode array covering 3.5 × 3.5 m³. (A) Eight days of detected and tracked EOD frequencies. Successfully tracked and validated signal traces of different fish are indicated in different colors. Signal traces that could not be clearly validated are indicated in white. Dark gray areas indicate night time, light gray areas day time. (B) Signal traces of three fish where the crossing EOD frequency traces of the upper two fish could reliably be resolved by the tracking algorithm. (C) Too many signal traces with similar frequencies compromise the tracking algorithm (670 − 672 Hz). Frequency peaks in PSDs belonging to multiple fish temporally overlay and prevent successful tracking.

Figure 10

Spatial behavior of a single A. macrostomus detected and tracked consecutively for 4 days. Heat-maps and contour lines show the fish's probability of presence across the monitored 3.5×3.5 m² area of the river during the night (top) and day (bottom). The observation area ranged from the river bank (x = 0) to the center of the river (x = 3.5) with similar extend in the flow direction of the river (see Figure 1C). Heat-maps of signal powers over electrodes are interpolated using a gaussian-kernel for illustrative purposes. Orange contour lines include the area in which the fish spends more than 50% of the time, the red lines more than 75% of the time respectively. Even though the fish certainly shows movement behaviors, especially during the night, it remains remarkably stationary in a specific location of the obervation area for four consecutive days.