Working memory contributions to reinforcement learning impairments in schizophrenia

Abstract

Previous research has shown that patients with schizophrenia are impaired in reinforcement learning tasks. However, behavioral learning curves in such tasks originate from the interaction of multiple neural processes, including the basal ganglia- and dopamine-dependent reinforcement learning (RL) system, but also prefrontal cortex-dependent cognitive strategies involving working memory (WM). Thus, it is unclear which specific system induces impairments in schizophrenia. We recently developed a task and computational model allowing us to separately assess the roles of RL (slow, cumulative learning) mechanisms versus WM (fast but capacity-limited) mechanisms in healthy adult human subjects. Here, we used this task to assess patients' specific sources of impairments in learning. In 15 separate blocks, subjects learned to pick one of three actions for stimuli. The number of stimuli to learn in each block varied from two to six, allowing us to separate influences of capacity-limited WM from the incremental RL system. As expected, both patients (n = 49) and healthy controls (n = 36) showed effects of set size and delay between stimulus repetitions, confirming the presence of working memory effects. Patients performed significantly worse than controls overall, but computational model fits and behavioral analyses indicate that these deficits could be entirely accounted for by changes in WM parameters (capacity and reliability), whereas RL processes were spared. These results suggest that the working memory system contributes strongly to learning impairments in schizophrenia.

Keywords: computational model; reinforcement learning; schizophrenia; working memory.

Figure 1.

Experimental protocol. At the beginning of each block, subjects were presented the set of stimuli they would have to learn the correct actions for in that block. Each trial included a 3.5 s presentation of a stimulus during which time subjects pressed one of three responses. Feedback indicating correct or incorrect followed. Block set sizes varied between two and six, and the order was randomized across subjects.

Figure 2.

Learning effects. Top, Learning curves per set size and group (SZ, patients; HC, healthy controls, dashed lines). Learning curves indicate the probability of a correct response at the nth presentation of a single stimulus (stim). Middle, Average performance for early trials (presentations 2 and 3 of each stimulus), for asymptotic trials (last 2 presentations of each stimulus), and over the whole block. Bottom, Reaction times (RT) for early trials, asymptotic trials, and the whole block. Error bars indicate SEM.

Figure 3.

RLWM model fit parameters. Parameters fit on subjects' behavior with the RLWM model, for the HC group and patients (SZ). Red indicates a significant difference between groups. Error bars indicate SEM. A, WM-specific parameter rates show less robust WM for patients (lower capacity N and faster decay rate φ_WM). B, RL-specific parameters (learning rate α and decay φ_RL) show no difference. C, Patients rely a priori less on WM (ρ_WM) and perseverate more than controls. D, Despite worse performance in patients, noise parameters show no difference between groups, indicating that this is attributable to deficit in learning more than deficit in choice. E, Absolute value of the weights on model parameters for the first two principal components. Blue bars indicate a positive weight and red bars a negative weight. The first component loads strongly on the two RL parameters, whereas the second component loads strongly on the four WM parameters. pers., Perseverate. F, The second component is correlated with an independent classic measure of working memory across all subjects (black regression line) and within each group.

Figure 4.

Learning curves and model simulations. Left, Empirical data per group (SZ, patients; HC, healthy controls, dashed lines). Middle, RLWM model simulation. Right, RL model simulation (δ rule learning model including undirected noise, forgetting, and perseveration mechanisms). Learning curves indicate the probability of a correct response at the nth presentation of a single stimulus (stim). For each subject's fitted parameters, one set of learning curves was obtained by averaging over 100 simulations of the model with those parameters. Overall learning curves were then obtained by averaging across subjects.