The temporal structure of goal-directed and habitual operant behavior

Abstract

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Keywords: bout analysis; context; goal-directed action; habit; operant.

FIGURE B1.

Posterior distribution of differences for Thrailkill & Bouton (2015). The left panel indicates the difference between training groups (${\mathrm{\beta }}_g$; brief vs. extended), the middle panel indicates the difference between contexts (${\mathrm{\beta }}_k$; Context A vs. Context B) for briefly trained animals, and the right panel indicates the difference between contexts (${\mathrm{\beta }}_{kg}$; Contexts A vs. Context B) for extended trained animals. Lines within each distribution indicate the 5th, 50th, and 95th quantile approximating the median and 95% credible interval of each posterior distribution. Parameters are as described for the right-hand side of Equation A2, where $H=H$ and thus $q=\frac{e^H}{1+e^H}$ and bout length is $1/q,w-bar=\bar{w}$ and thus $w=\frac{1}{\stackrel{-}{e^{\stackrel{-}{w}}}}$ and bout duration is $e^{\stackrel{-}{w}}(\frac{1}{q})$, and $b-bar=\stackrel{-}{b}$ and thus $b=\frac{1}{\stackrel{-}{e^w}}+\frac{1}{\stackrel{-}{e^b}}$ (omission of subscripts is because these estimates represent differences between groups and/or conditions, and thus the jth subject is not applicable, and then subscripts $k$ and $g$ are indicated by the effects under consideration; see panel titles and prior caption text).

FIGURE B2

Posterior distribution of differences for Bouton et al. (2020). Indicates the difference between feeding groups (${\mathrm{\beta }}_g$; not prefed vs. prefed). Lines within each distribution indicate the 5th, 50th, and 95th quantile approximating the median and 95% credible interval of each posterior distribution. Parameters are as described for the right-hand side of Equation A2, where $H=H$ and thus $q=\frac{e^H}{1+e^H}$ and bout length is $1/q,w-bar=\bar{w}$ and thus $w=\frac{1}{\stackrel{-}{e^{\stackrel{-}{w}}}}$ and bout duration is $e^{\stackrel{-}{w}}(\frac{1}{q})$, and $b-bar=\stackrel{-}{b}$ and thus $b=\frac{1}{\stackrel{-}{e^w}}+\frac{1}{\stackrel{-}{e^b}}$ (omission of subscripts is because these estimates represent differences between groups and/or conditions, and thus the jth subject isn’t applicable, and then subscripts $k$ and $g$ are indicated by the effects under consideration; see panel titles and prior caption text).

FIGURE B3

Posterior distribution of differences for Steinfeld and Bouton (2021). Indicates the difference between context conditions (${\mathrm{\beta }}_k$; habit vs. neutral context). Lines within each distribution indicate the 5th, 50th, and 95th quantile approximating the median and 95% credible interval of each posterior distribution. Parameters are as described for the right-hand side of Equation A2, where $H=H$ and thus $q=\frac{e^H}{1+e^H}$ and bout length is $1/q,w-bar=\bar{w}$ and thus $w=\frac{1}{\stackrel{-}{e^{\stackrel{-}{w}}}}$ and bout duration is $e^{\stackrel{-}{w}}(\frac{1}{q})$, and $b-bar=\stackrel{-}{b}$ and thus $b=\frac{1}{\stackrel{-}{e^w}}+\frac{1}{\stackrel{-}{e^b}}$ (omission of subscripts is because these estimates represent differences between groups and/or conditions, and thus the jth subject isn’t applicable, and then subscripts $k$ and $g$ are indicated by the effects under consideration; see panel titles and prior caption text).

FIGURE 1

Results from Thrailkill and Bouton (2015). Results are presented for unpaired groups in Context A (top) and Context B (bottom). Survivor log plots of IRTs (left) for group data in each test with corresponding line representing the mean fit of the biexponential refractory model (BERM). The right three panels show total response rate and mean BERM parameters estimates for individual subjects within each level of training group (extended and brief) and testing condition (Context A, B): bout-initiation $\left(b\right)$ and within-bout $\left(w\right)$ response rates (converted to responses/min), bout lengths $\left(L\right)$, and bout duration $\left(T_{Bout}\right)$. Error bars on box plots depict the range.

FIGURE 2

Results from Bouton et al. (2020). Results are presented for unpaired groups in the test session. Survivor log plots of IRTs (left) for group data in each test context, with corresponding line representing the fit of the biexponential refractory model (BERM). The right three panels show for individual subjects within each feeding group total response rate and mean BERM parameters estimates: bout-initiation $\left(b\right)$ and within-bout $\left(w\right)$ response rates (converted to responses/min), bout lengths $\left(L\right)$, and bout duration $\left(T_{Bout}\right)$. Error bars on box plots depict the range.

FIGURE 3

Results from Steinfeld and Bouton (2021). Results are presented for unpaired groups in the test in the neutral (renewal) context and the habit context. Survivor log plots of IRTs (left) for group data in each test context, with corresponding line representing the fit of the biexponential refractory model (BERM). The right three panels show for individual subjects within each context condition total response rate and mean BERM parameters estimates: bout-initiation $\left(b\right)$ and within-bout $\left(w\right)$ response rates (converted to responses/min), bout lengths $\left(L\right)$, and bout duration $\left(T_{Bout}\right)$. Error bars on box plots depict the range.