Reinforcement learning account of network reciprocity

Abstract

Evolutionary game theory predicts that cooperation in social dilemma games is promoted when agents are connected as a network. However, when networks are fixed over time, humans do not necessarily show enhanced mutual cooperation. Here we show that reinforcement learning (specifically, the so-called Bush-Mosteller model) approximately explains the experimentally observed network reciprocity and the lack thereof in a parameter region spanned by the benefit-to-cost ratio and the node's degree. Thus, we significantly extend previously obtained numerical results.

Fig 1. Ring networks composed of N = 20 players.

The player represented by a black circle is adjacent to k players represented by gray circles. (a) k = 2. (b) k = 6.

Fig 2. Fraction of cooperative players in each round, averaged over 10³ simulations.

We set k = 2, N = 100, t_max = 50, β = 0.2, A = 1.0, and ϵ = 0.05. (a) b/c = 6. (b) b/c = 2.

Fig 3. Fraction of cooperative players under the static-network treatment [(a) and (e)] and the shuffled-network treatment [(b) and (f)].

The difference between the fraction of cooperation in the static and shuffled networks is shown in (c) and (g). The assortment for the static networks is shown in (d) and (h). We set N = 100 and β = 0.2. (a)–(d) b/c = 2. (e)–(h) b/c = 6. To calculate the fraction of cooperators and the assortment, we take averages over the first 25 rounds and 10³ simulations.