Rating expectations can slow aversive reversal learning

Abstract

The process of learning allows organisms to develop predictions about outcomes in the environment, and learning is sensitive to both simple associations and higher order knowledge. However, it is unknown whether consciously attending to expectations shapes the learning process itself. Here, we directly tested whether rating expectations shapes arousal during classical conditioning. Participants performed an aversive learning paradigm wherein one image (CS+) was paired with shock on 50% of trials, while a second image (CS-) was never paired with shock. Halfway through the task, contingencies reversed. One group of participants rated the probability of upcoming shock on each trial, while the other group made no online ratings. We measured skin conductance response (SCR) evoked in response to the CS and used traditional analyses as well as quantitative models of reinforcement learning to test whether rating expectations influenced arousal and aversive reversal learning. Participants who provided online expectancy ratings displayed slower learning based on a hybrid model of adaptive learning and reduced reversal of SCR relative to those who did not rate expectations. Mediation analysis revealed that the effect of associative learning on SCR could be fully explained through its effects on subjective expectancy within the group who provided ratings. This suggests that the act of rating expectations reduces the speed of learning, likely through changes in attention, and that expectations directly influence arousal. Our findings indicate that higher order expectancy judgments can alter associative learning.

Keywords: aversive learning; conditioning; defensive; expectancy; fear; reinforcement learning; skin conductance; threat.

FIGURE 1

Task design. (a) Participants assigned to a Rating Group made expectancy ratings during aversive reversal learning, while Viewing Group participants viewed images and received shocks in the same task without making ratings. (b) There was a 50% reinforcement rate for the CS+ (i.e., 14 unreinforced presentations and 14 reinforced presentations). Halfway through the task, contingencies reversed and the initial CS− became the new CS+ and was reinforced with a 50% reinforcement rate for the duration of the task. (c) On each trial, the CS was presented for 4 s followed by a 12 s inter‐stimulus interval. CS+ presentation coterminated with a 200 ms shock. Two stimuli were used (purple/yellow fractal or red/green fractal) and initial CS assignment was counterbalanced across participants

FIGURE 2

Retrospective ratings. Upon task completion, participants retrospectively rated (a) number of perceived reversals, (b) probability of shock associated with each stimulus at the beginning and at the end of the study; and (c) affect in response to each stimulus. Groups did not differ in any retrospective ratings. Error bars denote SEM

FIGURE 3

Skin conductance as a function of Group and Phase. Left: This figure illustrates skin conductance responses as a function of Stimulus prior to reversal (top) and following contingency reversals (bottom). Only the Viewing Group showed significant differences prior to reversal and a complete reversal of the differential response when contingencies changed. Middle: SCR and expectancy ratings show a similar timecourse on unreinforced trials within Rating Group participants, where responses do not reverse until several trials after the reversals. Top right: SCR in the Viewing Group reverses immediately upon contingency reversal. Raincloud plots are visualized using the R package raincloudplots (Allen et al., 2021)

FIGURE 4

Hybrid model learning parameters differ by group. Fitting a Rescorla‐Wagner model of reinforcement learning to SCR on unreinforced trials revealed higher learning rates in Viewing Group participants than participants in the Rating Group, whether fit to individuals or jack‐knife estimation to iteratively fit across each group using cross validation. Learning rates depicted here are from jack‐knife estimation. See Figure S1 for complete results of jack‐knife estimation and fits to individuals

FIGURE 5

Expectancy fully mediates differential response within participants who make ratings. Multilevel mediation revealed that trial‐by‐trial expectancy ratings fully mediated effects of the current contingencies on SCR across participants. We used bootstrap estimation to determine the significance of the mediation effect (Shrout & Bolger, 2002). Slope plots depict individual estimates in blue lines, with the 95% confidence interval depicted in the gray shaded area that surrounds the overall group effect. Analyses were conducted in the Multilevel Mediation Moderation Toolbox (Atlas et al., ; Wager et al., 2009). Upper left: There was a significant effect of current CS contingencies on subjective expectancy (i.e., Path a in the mediation framework). Upper right: There was a significant effect of expectancy on SCR, controlling for current CS contingencies (i.e., Path b in the mediation framework). Lower panel: There was a significant direct effect of Current CS contingencies on SCR, which was non‐significant when controlling for expectancy rating. There was a significant negative association between Path A and Path B coefficients, which suggests mediation was driven primarily by within‐subjects effects