Reinforcement Learning Enables Resource Partitioning in Foraging Bats

Abstract

Every evening, from late spring to mid-summer, tens of thousands of hungry lactating female lesser long-nosed bats (Leptonycteris yerbabuenae) emerge from their roost and navigate over the Sonoran Desert, seeking for nectar and pollen [1, 2]. The bats roost in a huge maternal colony that is far from the foraging grounds but allows their pups to thermoregulate [3] while the mothers are foraging. Thus, the mothers have to fly tens of kilometers to the foraging sites-fields with thousands of Saguaro cacti [4, 5]. Once at the field, they must compete with many other bats over the same flowering cacti. Several solutions have been suggested for this classical foraging task of exploiting a resource composed of many renewable food sources whose locations are fixed. Some animals randomly visit the food sources [6], and some actively defend a restricted foraging territory [7-11] or use simple forms of learning, such as "win-stay lose-switch" strategy [12]. Many species have been suggested to follow a trapline, that is, to revisit the food sources in a repeating ordered manner [13-22]. We thus hypothesized that lesser long-nosed bats would visit cacti in a sequenced manner. Using miniature GPS devices, aerial imaging, and video recordings, we tracked the full movement of the bats and all of their visits to their natural food sources. Based on real data and evolutionary simulations, we argue that the bats use a reinforcement learning strategy that requires minimal memory to create small, non-overlapping cacti-cores and exploit nectar efficiently, without social communication.

Keywords: behavioral ecology; movement ecology; nectar feeding bats; reinforcement learning; resource partitioning; territories; trapline.

Figure 1

Typical Movement of Lactating Nectar-Feeding Bats (A) Bats fly tens of kilometers from the cave to their foraging site (colors represent different individuals; n = 11 bats for which we had at least one full night). The bat that flew farthest (red) flew a one-way distance of ~104 km on her first night (depicted by a black arrow) to her foraging site (white circle). The zoomed in circle panel shows how the cacti quality—the number of open flowers—differ greatly within one of the Saguaro fields, with 0–4 open flowers on an average night (0 flowers are presented as gray circles and 1–4 flowers are presented as blue to red circles). See also Figure S1. (B) Bats typically visited the same Saguaro field on consecutive nights (colors from blue to red depict different nights for each bat; three bats with a mean of 4.3 ± 2.0 nights per bat are shown). When returning to the same site, the bats visited ~75% of the cacti that they visited on the previous night (n = 8 bats; see also STAR Methods and Table S1).

Figure 2

Nectar-Feeding Bats Foraging Dynamics (A) Flight trajectory of one bat is shown (gray line), and the cacti it visited are colored according to the number of visits (unvisited cacti are not presented). Top left inset shows hops between cacti, where the width of the black lines represents the number of hops between cacti and colored circles represent only the core cacti that were visited more than 5 times at different hours throughout the night. In (B)–(F), colors are as follows: gray, real bats data; black, mean ± SE of 8 real bats; red, mean ± SE of 45 simulated bats (30 simulations). (B and C) The accumulated number of (B) visited cacti and (C) core cacti throughout the night (see also Figure S1J). (D) The proportion of visits per cactus are sorted according to cactus ID, i.e., cactus 1 is the cactus that received most visits. (E) The distribution of distances between consecutive visited cacti. (F) The distribution of time lags between revisits to the same cactus. Core cacti were close to each other (25.1 ± 13.6 m apart), and bats revisited them very often (consecutive visits to core cacti were on average 8.9 ± 3.5 min apart; E and F).

Figure 3

Comparison of Different Models (A–E) Foraging behavior of real bats (black, mean ± SE, n = 8 bats) and simulated bats (mean ± SE, n = 45 bats) using different foraging models: reinforcement learning (red); reinforcement learning with aggression (blue); trapline (yellow); and random (purple). (A and B) Accumulated number of (A) visited cacti and (B) core cacti (first 2.5 h). (C) Sorted proportion of visits per cactus (in the 20 most visited cacti). (D) The distribution of distances between consecutive visited cacti (30-m bins). (E) The distribution of time lags between revisits to the same cactus (first 20 min). (F) The fit of different foraging models (normalized mean distance from real data; data were normalized to a max of 1 before fitting). (G) Bats’ nectar consumption in different foraging models. (H) Core cacti of five simulated bats in the reinforcement learning model. Cacti of different individuals are depicted by different colors, and circle size represents the number of visits (accumulated over a full night). Cacti that were in the core of more than one bat appear in multiple colors according to the relative number of visits of each bat, and cacti that were in the core of a single bat are circled by a black line. See also Figure S2.