Longitudinal changes in reinforcement learning during smoking cessation: a computational analysis using a probabilistic reward task

Abstract

Despite progress in smoking reduction in the past several decades, cigarette smoking remains a significant public health concern world-wide, with many smokers attempting but ultimately failing to maintain abstinence. However, little is known about how decision-making evolves in quitting smokers. Based on preregistered hypotheses and analysis plan ( https://osf.io/yq5th ), we examined the evolution of reinforcement learning (RL), a key component of decision-making, in smokers during acute and extended nicotine abstinence. In a longitudinal, within-subject design, we used a probabilistic reward task (PRT) to assess RL in twenty smokers who successfully refrained from smoking for at least 30 days. We evaluated changes in reward-based decision-making using signal-detection analysis and five RL models across three sessions during 30 days of nicotine abstinence. Contrary to our preregistered hypothesis, punishment sensitivity emerged as the only parameter that changed during smoking cessation. While it is plausible that some changes in task performance could be attributed to task repetition effects, we observed a clear impact of the Nicotine Withdrawal Syndrome (NWS) on RL, and a dynamic relationship between craving and reward and punishment sensitivity over time, suggesting a significant recalibration of cognitive processes during abstinence. In this context, the heightened sensitivity to negative outcomes observed at the last session (30 days after quitting) compared to the previous sessions, may be interpreted as a cognitive adaptation aimed at fostering long-term abstinence. While further studies are needed to clarify the mechanisms underlying punishment sensitivity during nicotine abstinence, these results highlight the need for personalized treatment approaches tailored to individual needs.

Keywords: Decision-making; Nicotine abstinence; Punishment sensitivity; Reinforcement learning; Smoking cessation; Withdrawal.

Fig. 1

Study methods. (A) Study flowchart: task behavioral data from all participants (n = 110) were included in RL computational model fitting procedures. Statistical analysis was performed on 84 subjects: 19 completers who performed the task three times (S0, ABS2, and ABS30), 34 dropouts who performed the task twice (S1 and ABS2) and 31 healthy controls who performed the task once. (B) Probabilistic Reward Learning task structure.

Fig. 2

Abstinence-induced changes in measures of craving and withdrawal. Mean self-report measures of nicotine craving (TCQ) (A) and withdrawal symptoms (WSWS) (B) in completers by smoking status. Error bars represent standard errors. Black bars identify the significant differences (*p < 0.05, ** p < 0.01, *** P < 0.001) at pairwise comparison when the ANOVA was significant.

Fig. 3

Signal Detection Analyses. Top panel: Mean reward bias (A), discriminability (B), cumulative reward (C), fraction of correct trials (D), accuracy for rich trials (E) and accuracy for lean trials (F) in completers by smoking status. Bottom panel: Mean reaction time for lean and rich trials by accuracy and smoking status. Error bars represent standard errors.

Fig. 4

‘Punishment’ model output. Mean punishment sensitivity (log(β_punishment), (A), reward sensitivity (log(β_reward), (B), learning rate (log(α/(1-α)), (C), instruction sensitivity (log(γ), (D) and initial bias (log(q0), (E) in completers by smoking status. Error bars represent standard errors. Black bars identify the significant differences (*p < 0.05, ** p < 0.01, *** p < 0.001) at pairwise comparison when the ANOVA is significant. A dashed bar suggests that the difference does not survive when controlling for the others model’s parameters in the ANOVA.

Fig. 5

Relationship between smoking behavior and ‘Punishment’-based decision making. Top panel: change in Punishment Sensitivity (Δlog(β_punishment), (A), Reward Sensitivity (Δlog(β_reward), (B), and Learning Rate (Δlog(α/(1−α)), (C) in completers during extended abstinence (ABS30-S1). Smokers more severely addicted to nicotine (higher FTND) showed a greater increase in the parameters of interest. Bottom Panel: Correlation plots displaying relationships among Punishment Sensitivity (log(βpunishment)), Reward Sensitivity (log(βreward)), and Learning Rate (log(α/(1-α))) with Craving (TCQ scores) in completers, categorized by smoking status. Correlations at S0 and ABS2 were assessed using Pearson’s correlation (D–F), while correlations at ABS30 were evaluated using Spearman’s rank correlation (G–I).