Impaired value-based decision-making in Parkinson's disease apathy

Abstract

Apathy is a common and disabling complication of Parkinson's disease characterized by reduced goal-directed behaviour. Several studies have reported dysfunction within prefrontal cortical regions and projections from brainstem nuclei whose neuromodulators include dopamine, serotonin and noradrenaline. Work in animal and human neuroscience have confirmed contributions of these neuromodulators on aspects of motivated decision-making. Specifically, these neuromodulators have overlapping contributions to encoding the value of decisions, and influence whether to explore alternative courses of action or persist in an existing strategy to achieve a rewarding goal. Building upon this work, we hypothesized that apathy in Parkinson's disease should be associated with an impairment in value-based learning. Using a four-armed restless bandit reinforcement learning task, we studied decision-making in 75 volunteers; 53 patients with Parkinson's disease, with and without clinical apathy, and 22 age-matched healthy control subjects. Patients with apathy exhibited impaired ability to choose the highest value bandit. Task performance predicted an individual patient's apathy severity measured using the Lille Apathy Rating Scale (R = -0.46, P < 0.001). Computational modelling of the patient's choices confirmed the apathy group made decisions that were indifferent to the learnt value of the options, consistent with previous reports of reward insensitivity. Further analysis demonstrated a shift away from exploiting the highest value option and a reduction in perseveration, which also correlated with apathy scores (R = -0.5, P < 0.001). We went on to acquire functional MRI in 59 volunteers; a group of 19 patients with and 20 without apathy and 20 age-matched controls performing the Restless Bandit Task. Analysis of the functional MRI signal at the point of reward feedback confirmed diminished signal within ventromedial prefrontal cortex in Parkinson's disease, which was more marked in apathy, but not predictive of their individual apathy severity. Using a model-based categorization of choice type, decisions to explore lower value bandits in the apathy group activated prefrontal cortex to a similar degree to the age-matched controls. In contrast, Parkinson's patients without apathy demonstrated significantly increased activation across a distributed thalamo-cortical network. Enhanced activity in the thalamus predicted individual apathy severity across both patient groups and exhibited functional connectivity with dorsal anterior cingulate cortex and anterior insula. Given that task performance in patients without apathy was no different to the age-matched control subjects, we interpret the recruitment of this network as a possible compensatory mechanism, which compensates against symptomatic manifestation of apathy in Parkinson's disease.

Keywords: Parkinson’s disease; apathy; computational modelling; decision-making; functional MRI; reward insensitivity.

Figure 1

Restless Bandit Task. (A) Example of the underlying payout (reward) structure across the 300 trials of the task for each of the four bandits. The payout varied from one trial to the next by a Gaussian walk. (B) Each trial has a fixed trial length of 6 s with a variable inter-trial interval designated by the time between the fixation cross and the onset of the trial (mean 2 s). At trial onset, four coloured squares (bandits) were presented. The participant selected one bandit within 1.5 s, which was then highlighted by the bandit lever depressing and a chequer board appearing (choice screen). After a 3 s delay the, the outcome of the choice, as number of ‘points’ won, was displayed for 1 s. In trials where the participant failed to make a choice within 1.5 s, the choice screen was replaced by a large red cross (not shown) signifying a missed trial. (C) An example of a single subject’s choices in the task (top) fitted to a reinforcement learning model used to estimate the latent neural encoding of each bandits estimated value and uncertainty (variance). This model allows choices to be categorized into one of three decision types depending upon whether these were made to the bandit with the highest estimated value (exploitative choice) or to one of the three lower valued options (random exploration). Directed exploration choices (bottom) are to a lower valued bandit, which has least recently been selected and there is most uncertainty as to its true value.

Figure 2

Bandit performance and relationship with apathy severity. The average probability of choosing the bandit with the highest payout, P(Choose best bandit) is plotted in the three groups of participants. (A) The increase in average values of best bandit choice plotted in Trials 1, 5 and 10 confirm learning in all three groups from an initial random choice to above chance levels (horizontal dashed line). Best choice performance across six 50-trial blocks of the task was reduced in the PD-apathy group. Vertical lines = standard error of the mean (SEM). (B) Each circle represents average best bandit choice probability across the task for an individual subject with the horizontal and vertical bar represents the group mean and 95% confidence limits. (C) Apathy severity, measured by increasing LARS score (more positive values represent higher levels of apathy) correlated with individuals’ ability to choose the best bandit (rho = −0.43, P = 0.001). Performance in the task, as measured by the number of points won, also differed between groups and predicted apathy status (rho = −0.39 P = 0.003) in PD (D–F). Each group’s average reaction time over the six task bins in G and likelihood of not making a response (H) (missed trial) was not affected by apathy. HC = healthy controls; LARS = Lille Apathy Rating Scale; PD = Parkinson’s disease.

Figure 3

Decision types between groups across the task and relationship with apathy severity. (A–C) Probability of making an exploit P(Exploit) choice plotted across six 50-trial bins for each group tested (A). Error bars represent standard error of the mean (SEM). (B) Individual P(Exploit) across the whole task is represented by each circle. Group average and SEM is illustrated by the vertical and horizontal bars. Correlation between P(Exploit) and apathy severity (increasing LARS score) in C rho = −0.50, P < 0.001. The same analysis applied to the probability of making a random exploratory P(Explore) choice (D and E) and relationship with individual apathy severity (F) rho = 0.47, P < 0.001. P(DE) is the probability of making a directed exploratory choice (G–I) and P(Stay) the same choice on two consecutive trials (J and K). (L) Correlation between P(Stay) and apathy severity, rho = −0.47, P < 0.001. HC = healthy controls; LARS = Lille Apathy Rating Scale.

Figure 4

Functional MRI BOLD correlates of the outcome signal in PD-apathy and non-apathetic groups. (A) Peak activations in healthy control (HC) group at the point of feedback of the choice outcome (pay-off) time demonstrated a significant cluster in left ventromedial prefrontal cortex (vmPFC) after family-wise error (FWE) whole brain correction at P < 0.05 (T = 5.25, P_{cluster FWE WB} < 0.001). (B) Activations in PD-no apathy group did not survive correction at a whole brain level but were present in vmPFC when analysed with small volume correction using a region of interest analysis centred on the peak activation in the healthy control group (T = 4.51, P_{peak FWE SVC} = 0.004). No significant clusters activity survived whole brain or small volume correction in the PD-apathy group (C). Contrast analysis between groups did not demonstrate any difference in outcome signal activations between the PD-apathy and PD-no apathy groups (result not illustrated). However, combining the PD-no apathy and healthy control groups confirmed a significant reduction in the outcome signal in left vmPFC in the PD-apathy patients (D) (T = 4.60, P_{cluster FWE WB} = 0.01). BOLD = blood oxygen level-dependent; PD = Parkinson’s disease.

Figure 5

Functional MRI BOLD signal correlates at the decision time during exploratory choices. (A) Peak activations in healthy control group at the point of decision-making during exploratory choice activated parieto-occipital, thalamic and both medial (preSMA/dACC), lateral prefrontal (DLPFC) and bilateral anterior insula regions. Corresponding activation in PD-no apathy (B) and PD-apathy (C) groups. Significant clusters were defined as those surviving family-wise error whole brain correction at P < 0.05 (Supplementary Table 4). BOLD = blood oxygen level-dependent; PD = Parkinson’s disease.

Figure 6

Functional MRI BOLD contrast differences the decision time during exploratory choices. (A and B) Peak contrast difference in activation between the PD-apathy and PD no-apathy within the right thalamus at the point of decision-making during an exploratory choice. Blood oxygen level-dependent (BOLD) signal activations at this peak voxel plotted for each subject and group (C) and correlation with individual apathy severity (D). Functional connectivity analysis at this voxel using normative functional resting state connectivity showed peak correlation between the thalamus and dACC/bilateral AI (E). AI = anterior insula; dACC = dorsal anterior cingulate cortex; HC = healthy controls; LARS = Lille Apathy Rating Scale.