Rationalizing spatial exploration patterns of wild animals and humans through a temporal discounting framework

Abstract

Understanding the exploration patterns of foragers in the wild provides fundamental insight into animal behavior. Recent experimental evidence has demonstrated that path lengths (distances between consecutive turns) taken by foragers are well fitted by a power law distribution. Numerous theoretical contributions have posited that "Lévy random walks"-which can produce power law path length distributions-are optimal for memoryless agents searching a sparse reward landscape. It is unclear, however, whether such a strategy is efficient for cognitively complex agents, from wild animals to humans. Here, we developed a model to explain the emergence of apparent power law path length distributions in animals that can learn about their environments. In our model, the agent's goal during search is to build an internal model of the distribution of rewards in space that takes into account the cost of time to reach distant locations (i.e., temporally discounting rewards). For an agent with such a goal, we find that an optimal model of exploration in fact produces hyperbolic path lengths, which are well approximated by power laws. We then provide support for our model by showing that humans in a laboratory spatial exploration task search space systematically and modify their search patterns under a cost of time. In addition, we find that path length distributions in a large dataset obtained from free-ranging marine vertebrates are well described by our hyperbolic model. Thus, we provide a general theoretical framework for understanding spatial exploration patterns of cognitively complex foragers.

Keywords: Lévy walks; decision making; foraging theory; optimal search; temporal discounting.

Fig. S1.

Optimal search of a bounded area containing reward by a forager with spatial memory. A forager with spatial memory searches a bounded area containing uniformly distributed rewards. The optimal solution, assuming that the forager has a limited perceptual range, is to tessellate the search region into bins defined by the perceptual range so that every location in the search space can be sensed by single visits to the locations x_min + r_p, x_min + 3r_p, x_min + 5r_p, … x_max. If the forager has no spatial memory and hence cannot remember which location it visited on the previous search bout, it will be suboptimal.

Fig. 1.

Model for adaptive benefits of apparent Lévy walks. (A) The environments of foragers often show spatial autocorrelation with a mean spatial scale. (B and C) Thus, it is likely that foragers attempt to build a model of the mean spatial scale by building a model of rewards obtained for different flight distances (main text). (D) An optimal model of exploration that maximizes discriminability requires foragers to sample different flight distances (or durations) in proportion to the uncertainty in subjective value of rewards predicted at those distances. If the prior expectation of rewards is uniform, the sampling of different flight distances will produce a hyperbolic-like distribution (SI Results, 2.1) Optimal Exploration of Reward Distributions Across Relative Space)—due to hyperbolic discounting—that can appear to be power law distributed (Fig. 2).

Fig. S2.

Prey truncation leads to exponential path lengths when the density of food is high. Observed path lengths of foragers no longer reflect the intended path lengths when the distribution of prey is high due to prey-encounter truncation. The resultant path lengths can be shown to be exponential for a 2D environment with randomly distributed prey (SI Results, 2.4) Truncation Due to Prey Encounter).

Fig. 2.

Human spatial exploration task with and without temporal costs. (A) Schematic of the computer task (Methods and SI Methods). In phase 1, an albatross flies across the screen from a nest at a constant speed. In phase 2, subjects can make the albatross teleport (i.e., flight time is negligible). (B) Data from example subjects showing systematic search behavior across space. (C) Raw cumulative distribution function (CDF) of the population data across subjects for phase 1 and phase 2, showing sensitivity of exploration to the cost of time.

Fig. 3.

Previously collected data from wild animals are better fitted by a hyperbolic model than by a power law model. (A) Random data generated from a hyperbolic distribution (blue circles) can be well approximated by a power law distribution (red), but not by an exponential distribution (green): 25,000 random numbers were generated from a hyperbolic distribution (Eq. 2) (Methods) with truncation set to be between 10 and 1,000 (see Fig. S4 for more parameters). The best fit truncated power law describes the data significantly better than the best fit exponential (wAIC_tp = 1.000 and wAIC_exp = 0.000). (B) Previously collected data (blue circles) that are well fitted by a hyperbolic model (cyan) and a power law model (red), but not by an exponential model (green) (see Fig. S5 for fits across many subjects). (C) Data from a subject in which the hyperbolic model is considerably preferred to any alternative model.

Fig. S5.

Hyperbolic, power law, and exponential fits to all eight individual subjects shown in Table 1. The first row shows individuals 1 and 2, the second row shows individuals 3 and 4, etc. Data are shown in blue. Exponential fits are shown in green, power law fits in red, and hyperbolic fits in cyan.

Fig. S3.

Search pattern for all 12 human subjects for phase 1 and phase 2. Data from phases 1 and 2 are shown in brown and orange, respectively.

Fig. S4.

Simulation shown in Fig. 3A repeated for more parameters. Truncation of random data generated from a truncated hyperbolic distribution was set to x_min = 10 and x_max = 1,000. The AIC strongly favored the truncated power law fit over exponential (wAIC_tp = 1.000 and wAIC_exp = 0.000) in all cases.

Fig. 4.

Optimal exploration when temporal representations are noisy. (A) Optimal algorithm for exploration. (B) If temporal resolution is constant at every interval, subjective representation of time can be represented as a linear function of real time, with constant noise. However, it is known that errors in timing increase with the interval being timed (42). In this case, subjective representation of time can be represented as a nonlinear function with the nonlinearity controlled by the parameter T_ime (–45). (C) When subjective representation of time is nonlinear (concave), equal bins in subjective time correspond to bins of increasing width in real time. (D) A theory of reward rate maximization (–45) predicts linear sampling for optimal exploration in subjective time, with the slope determined by T_ime. In real time, this sampling becomes hyperbolic (Eq. 3) with its decay controlled by T_ime (SI Results, 2.1.5) Exploration under noisy temporal estimation).

Fig. S6.

Optimal exponents for power law fits to Eq. S8. If path length data conformed to Eq. 3 from the main text (derived for uncertainty in time perception), the closest approximation with a power law fit would have exponents as shown here. This is a numerical solution of Eq. S9 (SI Results, Eq. S9 (2.1.5) Exploration under noisy temporal estimation).