Models of Dynamic Belief Updating in Psychosis-A Review Across Different Computational Approaches

Abstract

To understand the dysfunctional mechanisms underlying maladaptive reasoning of psychosis, computational models of decision making have widely been applied over the past decade. Thereby, a particular focus has been on the degree to which beliefs are updated based on new evidence, expressed by the learning rate in computational models. Higher order beliefs about the stability of the environment can determine the attribution of meaningfulness to events that deviate from existing beliefs by interpreting these either as noise or as true systematic changes (volatility). Both, the inappropriate downplaying of important changes as noise (belief update too low) as well as the overly flexible adaptation to random events (belief update too high) were theoretically and empirically linked to symptoms of psychosis. Whereas models with fixed learning rates fail to adjust learning in reaction to dynamic changes, increasingly complex learning models have been adopted in samples with clinical and subclinical psychosis lately. These ranged from advanced reinforcement learning models, over fully Bayesian belief updating models to approximations of fully Bayesian models with hierarchical learning or change point detection algorithms. It remains difficult to draw comparisons across findings of learning alterations in psychosis modeled by different approaches e.g., the Hierarchical Gaussian Filter and change point detection. Therefore, this review aims to summarize and compare computational definitions and findings of dynamic belief updating without perceptual ambiguity in (sub)clinical psychosis across these different mathematical approaches. There was strong heterogeneity in tasks and samples. Overall, individuals with schizophrenia and delusion-proneness showed lower behavioral performance linked to failed differentiation between uninformative noise and environmental change. This was indicated by increased belief updating and an overestimation of volatility, which was associated with cognitive deficits. Correlational evidence for computational mechanisms and positive symptoms is still sparse and might diverge from the group finding of instable beliefs. Based on the reviewed studies, we highlight some aspects to be considered to advance the field with regard to task design, modeling approach, and inclusion of participants across the psychosis spectrum. Taken together, our review shows that computational psychiatry offers powerful tools to advance our mechanistic insights into the cognitive anatomy of psychotic experiences.

Keywords: Bayesian learning; Hierarchical Gaussian Filter; belief updating; change point detection; computational psychiatry; psychosis; reinforcement learning; schizophrenia.

Figure 1

Reversal learning task example and the associated HGF learning trajectories fitted to binary choice data of an individual participant. Upper plot: Depiction of trial sequence in a volatile reversal learning task with geometric stimuli (rewarded in this trial) [as in (22)]. Participants had to make a binary choice between one out of two stimuli via a button press and were presented with either a reward or loss outcome. Lower plot: Upper panel: Blue line represents one subject's individual trajectory of higher-level belief μ₃ over the course of the task from trial 1–160. Lower panel: Underlying contingencies are depicted in light brown with anti-correlated reward probabilities of one of the stimuli, reward contingencies remain stable in the beginning and end of the task with a volatile reversal period in between. Red line represents the belief μ₂ and reflects the tendency x₂ (or probabilistic strength) of stimulus A leading to reward. Black line represents the dynamic learning rate on the second level belief μ₂.

Figure 2

Example of the Helicopter paradigm (12) and the associated CPD learning trajectories. Upper plot: Depiction of trial sequence in the helicopter task, in which a hidden helicopter moves horizontally and drops a bucket in each trial. Participants have to give a continuous prediction of the bucket location via joystick which is followed by feedback with a visualized prediction error (the distance between their prediction and the actual bucket location in red). Middle plot: Light green represents the optimal trajectory of change-point probability and dark green represents the optimal trajectory of change-point probability (CPP) over the course of 140 trials. Lower plot: Dots represent the helicopter's locations, dispersing around the true mean (dashed line) in the low noise block (light brown) and the high noise block (light blue). Y-axis corresponds to the horizontal scale in the upper part.