Computational theory-driven studies of reinforcement learning and decision-making in addiction: What have we learned?

Abstract

Computational psychiatry provides a powerful new approach for linking the behavioral manifestations of addiction to their precise cognitive and neurobiological substrates. However, this emerging area of research is still limited in important ways. While research has identified features of reinforcement learning and decision-making in substance users that differ from health, less emphasis has been placed on capturing addiction cycles/states dynamically, within-person. In addition, the focus on few behavioral variables at a time has precluded more detailed consideration of related processes and heterogeneous clinical profiles. We propose that a longitudinal and multidimensional examination of value-based processes, a type of dynamic "computational fingerprint", will provide a more complete understanding of addiction as well as aid in developing better tailored and timed interventions.

Figure 1.

Computational “fingerprinting” and dynamic characterization of addiction trajectories and transitions. (A) The computational parameter space (a type of “computational fingerprint”) of a healthy individual showing select value-based decision-making parameters reported in the reviewed studies as being altered in addiction. The green shaded area represents a “healthy norm”. (B) Fluctuations over time of the computational fingerprints for prognosis-based addiction classification, i.e. recovering, cycling (abstinence and relapse stages), and sustained use cases as compared to health (green). (C) Evolution of the parameter space over time shown here for three components of the fingerprint for illustrative purposes (the full space may contain additional components). The example cases represent realistic trajectories/states: 1) healthy, shown to stay at the same multidimensional space over time; 2) sustained use, also shown to stay in the same space over time but to occupy a different one from health; 3) cycling use, shown to move away from an initial starting point and then start to return back to it. Here we also highlight at what time points tailored treatment might be most efficacious (i.e. when individuals might be most susceptible to intervention strategies) designated by the solid arrows (→); and 4) recovering, also shown to move but in a single direction approaching health. Note that here “component” could be a single estimated parameter (as shown in the 3D plot), a single estimated parameter accounting for the influence of another parameter (e.g., risk-preference adjusted learning rate), or a principal component (dimension comprised of a combination of parameters).