Foraging as an evidence accumulation process

Abstract

The patch-leaving problem is a canonical foraging task, in which a forager must decide to leave a current resource in search for another. Theoretical work has derived optimal strategies for when to leave a patch, and experiments have tested for conditions where animals do or do not follow an optimal strategy. Nevertheless, models of patch-leaving decisions do not consider the imperfect and noisy sampling process through which an animal gathers information, and how this process is constrained by neurobiological mechanisms. In this theoretical study, we formulate an evidence accumulation model of patch-leaving decisions where the animal averages over noisy measurements to estimate the state of the current patch and the overall environment. We solve the model for conditions where foraging decisions are optimal and equivalent to the marginal value theorem, and perform simulations to analyze deviations from optimal when these conditions are not met. By adjusting the drift rate and decision threshold, the model can represent different "strategies", for example an incremental, decremental, or counting strategy. These strategies yield identical decisions in the limiting case but differ in how patch residence times adapt when the foraging environment is uncertain. To describe sub-optimal decisions, we introduce an energy-dependent marginal utility function that predicts longer than optimal patch residence times when food is plentiful. Our model provides a quantitative connection between ecological models of foraging behavior and evidence accumulation models of decision making. Moreover, it provides a theoretical framework for potential experiments which seek to identify neural circuits underlying patch-leaving decisions.

Fig 1. Foraging-drift-diffusion model.

(A) Schematic showing the patch-leaving task: A forager estimates the average rate of reward from the environment, and the decision to leave a patch occurs when the internal decision variable reaches a threshold. Travel time between patches is T_tr, and patches are described by the parameters ρ₀, A, and c (see Table 2). (B) Evolution of the probability density of the patch decision variable (x) while in a single patch, along with the time-dependent probability that the decision to leave the patch has been made. Blue arrows denote the receipt of food rewards. (C) Energy estimate coupled with the patch decision variable over multiple patches. (D) Patch depletion with discrete rewards, showing examples of the food reward received and the time-dependent in-patch food density for different values of the food chunk size (c).

Fig 2. Patch-leaving decision strategies.

Different strategies are represented with different choices of the drift rate (α) and the threshold (η) (Eq 10). (A) The optimal threshold from Eq 9 plotted as a function of the drift, showing the general classes of increment-decrement and decremental strategies. The solid line shows the optimal threshold in the region of parameter space where both α and η have the same sign, and the dotted line indicates the region where they have the opposite sign. Parameters used are A = 5, E* = 2 (or equivalently, ρ₀ = 9.439), and s = 2; these parameters are also used in (B-E), which illustrate each strategy using discrete rewards and zero noise on the decision variable. (B) The choice α = ρ₀ is optimal for uncertainty in patch food density; this represents an “increment-decrement” mechanism for patch decisions. (C) A threshold of zero is optimal for uncertainty in patch size. Since η = 0 is sensitive to noise, we choose a small value η > 0 to illustrate. (D) The counting strategy uses zero drift, so that the forager leaves after a set amount of food rewards (E) The robust counting strategy uses α < 0 so that there is still drift towards the threshold. Each plot shows the patch decision variable along with the time-dependent patch decision threshold that changes with receipt of food reward due to updates of energy estimate.

Fig 3. Noisy evidence accumulation and discrete food rewards.

Shown are the average and standard deviation of the energy intake and patch residence times, simulated using intermediate values of the patch parameters: A = 5, T_tr = 5, and E* = 2 (or equivalently, ρ₀ = 9.439). The filled blue curves use the density-adaptive strategy, and the filled orange curves use the robust counting strategy. The robust counting strategy simulations use α = −0.2ρ₀. (A) Simulation results when the noise on the patch decision variable (σ) is increased. (B) Simulation results when the food chunk size (c) is increased. Analogous simulation results for a range of different parameter values (see Methods) are shown in S1 and S2 Figs.

Fig 4. Different foraging environments with associated patch decision strategies.

Shown are simulation results with the density-adaptive and robust-counting strategies in two different foraging environments. (A,D) illustrates the foraging environment for a given case, (B,E) shows average energy and patch residence time when a particular strategy is used in that environment along with the MVT-optimal energy energy (E*) and patch residence time (T*), and (C,F) shows simulation results compared to MVT-optimal strategies in each environment. All simulations use a noise level of $\sigma =0.3\overline{{\rho }_0}$ and a patch size of A = 5, and the robust counting strategy is implemented by setting α = −0.2ρ₀. (A-C) Uncertainty in patch food density. Patches have a Gaussian distribution for initial food density with mean of $\overline{{\rho }_0}=9.439$ and a standard deviation of $\Delta {\rho }_0=0.3\overline{{\rho }_0}$, and rewards are received continuously (c = 0). Travel time between patches is constant at T_tr = 5. The solid line in (C) shows an approximate analytical solution (Methods, Eq 25) for small changes in ρ₀ about $\overline{{\rho }_0}$. (D-F) Scattered patches with discrete rewards. Food reward is received in discrete chunks (c = 8) and each patch has the same initial food density of ρ₀ = 9.439. Travel time between patches is drawn from an exponential distribution with mean $\overline{T_{tr}}=5$. (F) shows a histogram of simulation results for how many food chunks were taken before leaving the patch, for both the density adaptive strategy (left) and the robust counting strategy (right).

Fig 5. Sub-optimal behavior.

The marginal utility of additional food reward may depend on the current rate of energy intake. We consider two possible functions: (A) Exponential decreasing utility, shown using A = 0 in Eq 13. (B) Threshold linear decreasing utility (Eq 14), shown here using a threshold of 0.65. Each form of the utility function has a parameter β that sets how fast the utility decreases with energy. Simulation results using the exponential utility function are shown in (C), and corresponding results using the threshold linear utility function in (D). For each case of the utility function, the average energy intake and patch residence time are shown for two different values of β. Solid lines are an approximate solution to the governing equations and points are the mean and standard deviation of simulation results. Both (C) and (D) use the density-adaptive strategy, and an environmental configuration where patch food density is uncertain (i.e. the same configuration and parameters as Fig 4A–4C). Analogous results for the robust counting strategy, and an additional environmental configuration, are shown in S4 Fig.