Perceptual interventions ameliorate statistical discrimination in learning agents

Abstract

Choosing social partners is a potentially demanding task which involves paying attention to the right information while disregarding salient but possibly irrelevant features. The resultant trade-off between cost of evaluation and quality of decisions can lead to undesired bias. Information-processing abilities mediate this trade-off, where individuals with higher ability choose better partners leading to higher performance. By altering the salience of features, technology can modulate the effect of information-processing limits, potentially increasing or decreasing undesired biases. Here, we use game theory and multiagent reinforcement learning to investigate how undesired biases emerge, and how a technological layer (in the form of a perceptual intervention) between individuals and their environment can ameliorate such biases. Our results show that a perceptual intervention designed to increase the salience of outcome-relevant features can reduce bias in agents making partner choice decisions. Individuals learning with a perceptual intervention showed less bias due to decreased reliance on features that only spuriously correlate with behavior. Mechanistically, the perceptual intervention effectively increased the information-processing abilities of the individuals. Our results highlight the benefit of using multiagent reinforcement learning to model theoretically grounded social behaviors, particularly when real-world complexity prohibits fully analytical approaches.

Keywords: partner choice; perceptual interventions; reinforcement learning; statistical discrimination.

Fig. 1.

The stages of iterative partner choice: During free mixing, individuals have access to observable features, but not directly their suitability as partners. Individuals form pairs based on what information is available to them and proceed to interact. The outcome of the interaction is a pair of payoffs (or rewards) assigned to the individuals in each of the pairs.

Fig. 2.

The boat race environment. (A) Six players (teal and purple) pair up to row boats (brown) and cross the river to access apples (green) that confer rewards. Agents can move freely on the river banks (black) but cannot walk across the river (water sprites in blue). Access to the boats is gated by barriers (gray). On the Left, we see a frame from before a race starts (semaphores are red) and agents are able to move freely. On the Right, we see a frame from after the race has started. The semaphores have turned green, and the barriers lifted to provide access to the boats. (B) Avatars are assigned a unique random “badge” at the beginning of an episode so their identity can be known throughout the episode.

Fig. 3.

Focal agent’s discrimination index $\mathcal{D}$ in their evaluation communities. Positive values indicate choosing partners based on their behavior, negative ones indicate choosing based on their color. (A) ${\mathcal{D}}_7$ in evaluation communities as a function of training bias. The higher the training bias, the worse the discrimination index is. On the Left, the baseline shows negative ${\mathcal{D}}_7$ overall. On the Right, the strongest perceptual intervention shows positive ${\mathcal{D}}_7$ overall. (B) $\mathcal{D}$ averaged over training biases is shown for the experimental conditions: no memory, baseline, and four levels of decay β of the perceptual intervention. Stronger perceptual interventions (those with less decay) result in improved ${\mathcal{D}}_7$. CIs are computed by bootstrapping each agent’s discrimination index (5,000 samples).

Fig. 4.

(A and B) The empirical values of awareness and stickiness for the focal agents, respectively. (C) The discrimination index for the analytical model, computed from $P_j^{(8)}$. Each figure shows a contour plot of the average discrimination index across all color biases (B) in terms of the stickiness s (on the x-axis) and awareness ω (on the y-axis). The empirical stickiness and awareness of the baseline agents and those trained with perceptual interventions are shown overlaid on the graph. (D) The average episode reward of the focal agents. Error bars correspond to SE.