Exploration dynamics in evolutionary games

Abstract

Evolutionary game theory describes systems where individual success is based on the interaction with others. We consider a system in which players unconditionally imitate more successful strategies but sometimes also explore the available strategies at random. Most research has focused on how strategies spread via genetic reproduction or cultural imitation, but random exploration of the available set of strategies has received less attention so far. In genetic settings, the latter corresponds to mutations in the DNA, whereas in cultural evolution, it describes individuals experimenting with new behaviors. Genetic mutations typically occur with very small probabilities, but random exploration of available strategies in behavioral experiments is common. We term this phenomenon "exploration dynamics" to contrast it with the traditional focus on imitation. As an illustrative example of the emerging evolutionary dynamics, we consider a public goods game with cooperators and defectors and add punishers and the option to abstain from the enterprise in further scenarios. For small mutation rates, cooperation (and punishment) is possible only if interactions are voluntary, whereas moderate mutation rates can lead to high levels of cooperation even in compulsory public goods games. This phenomenon is investigated through numerical simulations and analytical approximations.

Fig. 1.

Dynamics of the system with cooperators, defectors and punishers in the simplex S₃ for different mutation rates. The arrows show the drift term 𝒜_k(x) of the Fokker–Planck equation, white circles are stable fixed points in the limit M → ∞. The discontinuities are a consequence of the strong selection. Blue corresponds to fast dynamics and red to slow dynamics close to the fixed points of the system. The system does typically not access the gray shaded area, because the minimum average fraction of each type because of mutations is μ/3. (A) For vanishing mutation probability (μ → 0), there is only 1 stable fixed point in the defector corner. (B) For μ = 0.2, there are 2 stable fixed points, one close to the cooperator corner and one close to the defector corner. The population noise can drive the system from the vicinity of one of these points to the other, which makes an analytical description of the dynamics difficult. (C) For μ = 0.5, there is only a single stable fixed point, which is closest to the cooperator corner—thus, cooperators prevail for high mutation rates. We use the position of this fixed point as an estimate for the average abundance of the strategies, see Fig. 2 [parameters: M = 100, N = 5, r = 3, c = 1; β = 1, γ = 0.3; σ = 1, graphical output based on the Dynamo software (37)].

Fig. 2.

Imitation dynamics for different mutation rates. Symbols indicate results from individual-based simulations (averages over 10⁹ imitation steps), and solid lines show the numerical solution of the Fokker–Planck-equation, corresponding to a vanishing drift term 𝒜_k(x), see SI Appendix. Because a fraction μ of the population always mutates, the minimum fraction of each type is μ/d (for d strategies) and the gray shaded areas are inaccessible to the process. Although previous approaches have focused on small mutation rates, large mutation rates change the outcome significantly. (A) In compulsory public goods interactions, defectors dominate cooperators for all mutation rates. (B) In compulsory public goods interactions with punishment, defectors only dominate for small μ. For high mutation (or exploration) rates μ, cooperators dominate, see Fig. 1 for details. Despite their small abundance, punishers are pivotal for the large payoff and high abundance of cooperators for μ > 0.2. (C) In voluntary public goods games, cooperators dominate for high mutation rates as well. Although this effect does not depend on the presence of loners, these are essential for the success of punishers for small mutations. The horizontal lines for small μ, to which the symbols converge for μ → 0, is the stationary solution of the Markov chain, see SI Appendix (parameters: M = 100, N = 5, r = 3, c = 1; β = 1, γ = 0.3; σ = 1).