Collective cooperative intelligence

Abstract

Cooperation at scale is critical for achieving a sustainable future for humanity. However, achieving collective, cooperative behavior-in which intelligent actors in complex environments jointly improve their well-being-remains poorly understood. Complex systems science (CSS) provides a rich understanding of collective phenomena, the evolution of cooperation, and the institutions that can sustain both. Yet, much of the theory in this area fails to fully consider individual-level complexity and environmental context-largely for the sake of tractability and because it has not been clear how to do so rigorously. These elements are well captured in multiagent reinforcement learning (MARL), which has recently put focus on cooperative (artificial) intelligence. However, typical MARL simulations can be computationally expensive and challenging to interpret. In this perspective, we propose that bridging CSS and MARL affords new directions forward. Both fields can complement each other in their goals, methods, and scope. MARL offers CSS concrete ways to formalize cognitive processes in dynamic environments. CSS offers MARL improved qualitative insight into emergent collective phenomena. We see this approach as providing the necessary foundations for a proper science of collective, cooperative intelligence. We highlight work that is already heading in this direction and discuss concrete steps for future research.

Keywords: collective action; complex systems science; cooperation; multiagent reinforcement learning.

Fig. 1.

Multistability in CRLD (Box 1) applied to the ecological tipping environment (Box 2). (A) Phase space of the prosperous state. (B) Detailed bundle of learning trajectories. (C) Emergent timescales with abrupt transitions in transient dynamics. (D) Critical slowing down around the critical point. See SI Appendix for details.

Fig. 2.

Critical transitions in CRLD (Box 1) applied to the ecological tipping environment (Box 2). (A) Cooperation levels and final rewards. (B) Time steps to convergence show a critical slowing down around the critical point. For each discount factor, both plots show a histogram of converged results from 250 random initial conditions by a color map, their mean by large markers, their median by small markers, and the range between lower and upper quartiles by the shaded regions. See SI Appendix for details.

Fig. 3.

Hysteresis in CRLD (Box 1) applied to the ecological tipping environment (Box 2). The discount factor changes while the agents keep on learning. The size and color of the dots represent the time, from dark to light and from big to small as from past to future. See SI Appendix for details.