Parallel temporal dynamics in hierarchical cognitive control

Abstract

Cognitive control allows us to follow abstract rules in order to choose appropriate responses given our desired outcomes. Cognitive control is often conceptualized as a hierarchical decision process, wherein decisions made at higher, more abstract levels of control asymmetrically influence lower-level decisions. These influences could evolve sequentially across multiple levels of a hierarchical decision, consistent with much prior evidence for central bottlenecks and seriality in decision-making processes. However, here, we show that multiple levels of hierarchical cognitive control are processed primarily in parallel. Human participants selected responses to stimuli using a complex, multiply contingent (third order) rule structure. A response deadline procedure allowed assessment of the accuracy and timing of decisions made at each level of the hierarchy. In contrast to a serial decision process, error rates across levels of the decision mostly declined simultaneously and at identical rates, with only a slight tendency to complete the highest level decision first. Simulations with a biologically plausible neural network model demonstrate how such parallel processing could emerge from a previously developed hierarchically nested frontostriatal architecture. Our results support a parallel processing model of cognitive control, in which uncertainty on multiple levels of a decision is reduced simultaneously.

Keywords: Basal ganglia; Computational model; Executive function; Prefrontal cortex; Serial vs. parallel.

Figure A.1. Order of rule learning

Average number of errors made for individual rules on (a) the middle level of the hierarchy and (b) the low level of the hierarchy. Error bars depict standard error.

Figure C.1. Model simulations

The model successfully recovered the parameters underlying the mock data sets, for a variety of temporal patterns, some of which are pictured here. The graphs above illustrate the parameters that were selected to create the mock data sets (in the “Generative” column), as well as the parameters that were recovered by the model (in the “Recovered” column). Each graph shows how the probability of making an error on each level of the decision changes across time in a trial, as well as the changing probability of making a correct response. Some of the temporal patterns tested include: (a & b) serial: error rates decrease for one level at a time, (c) partially parallel: each level has a different start and end, but there is some overlap in the resolution of uncertainty, (d) equal start: error rates for all levels start to decrease at the same time, (e) equal end: error rates on all levels reach asymptote at the same time, (f) asymptotic differences: error rates are different between levels at the end of the trial, (g) partially parallel: the top level resolves before the middle and low levels, which have the same start time, and (h) middle before top: the error rate for the middle level of the decision starts to decrease first.

Figure D.1. Experiential effects on group-level error rates

These plots show how error rates for each policy level change over the window of time sampled in three subsections of the experiment: (a) the response deadline practice phase, (b) the first two experimental blocks, and (c) the fifth and sixth experimental blocks. All participants were included in this analysis, and trials were pooled between individuals (i.e., error rates are on the group level). Responses are binned by deadline and plotted by average response time (measured from the onset of the stimulus).

Figure E.1. Error rate switch cost

Switch costs were calculated by subtracting the error rate in a repeat trial (all rules repeat) from the error rate in each type of switch trial. Error bars depict standard error.

Figure E.2. Rule-switching effects on group-level error rates

This analysis includes the 18 participants who were presented with the final set of deadlines. Responses were pooled across the group, binned by deadline, and plotted by average response time (measured from the onset of the stimulus).

Figure 1. Serial reduction of uncertainty

When each level of a hierarchical decision is resolved, it reduces uncertainty on all lower levels. In the three-level binary decision tree shown, solving a level reduces the number of relevant items on all lower levels by half. Potentially relevant paths are shown in black for each stage of the serial decision, while irrelevant paths are gray.

Figure 2. Experimental task

(a) The experiment involved a stimulus-response task with rules that were structured hierarchically. The three levels of hierarchy are labeled in this figure (top, middle, and low). The stimuli for the task had seven dimensions, named in boxes. Each dimension had two variants (e.g. stripes and patterns), which either indicated which lower-level dimension was relevant, or mapped directly to a response. The responses are numbered 1-8 at the bottom of the tree and correspond to buttons pressed using the fingers shown in (b).

Figure 3. Order of rule learning

Participants learned and practiced the rule set incrementally and in a prescribed order.

Figure 4

Each level of the decision, examined independently from the others, reduces the number of responses that are potentially correct (from eight to four). In this figure, the responses that are consistent with each level of the hierarchy are circled (for the example stimulus, pictured on a black background). Only one of the eight responses is consistent with all three levels of the rule set (button 2): this is the correct response.

Figure 5

These plots show how (a) accuracy and (b) error rates for each policy level change over the window of time sampled. These analyses only include the 18 participants who were presented with the final set of deadlines. In both graphs, responses are binned by deadline and plotted by average response time (measured from the onset of the stimulus). Error bars show standard error.

Figure 6

The asymptotic accuracy calculated from the model results was highly correlated to the asymptotic accuracy calculated from the raw behavioral data (r=0.97).

Figure 7. Results from MCMC modeling

The mean and standard deviation of the distributions for the group parameters are shown above, represented graphically (a) and in a table (b).

Figure 8. Emergent network

Modified version of the basal ganglia loop model from Collins & Frank (2013), shown in an exemplar state of activity. Layers are labeled and represented as bordered rectangles containing circular units. Cylinders indicate the strength of a unit's activity at a particular moment. Our model manipulations affected the strength of the connection from PFC to Striatum and the connection from PC to Striatum. PC: Parietal cortex; PFC: Prefrontal cortex; PMC: Premotor cortex; SNc: Substancia nigra pars compacta; STN: Subthalamic nucleus; GP_Int: globus pallidus internal segment; GP_Ext: globus pallidus external segment. Layers labeled “2” indicate the higher-order loop.

Figure 9. Task for Emergent simulations

A schematic of the task that the network learned, with two levels of hierarchy (top and low), three features (shape, orientation, and location) and four possible responses (R1-R4).

Figure 10. Emergent results plotted as trajectories of each response type

Support for each response was calculated from striatal Go and NoGo activity (see equation 2) averaged over five runs of the network (i.e. five initializations of the weights). Each run consisted of five epochs (40 trials). Inset illustrates what each response trajectory would correspond to on a decision tree for an example trial of the experiment. (a) Trajectory of response support over time using the default network weights. In the default settings, the weight of the connection from PFC to Striatum was less than the weight of the projection from PC to Striatum. (b) Trajectory of response support over time when there is less influence of PFC on Striatum. The relative weight of the projection from PFC to Striatum was decreased from 0.75 to 0.5, while the relative weight of the projection from PC to Striatum remained at the default level of 1.25. (c) Trajectory of response support over time when there is the least influence of PFC on Striatum. The relative weight of the projection from PFC to Striatum was decreased to 0.3, while the relative weight of the projection from PC to Striatum was kept at the default level of 1.25. (d) Trajectory of response support over time when there is equal PFC and PC influence on Striatum. The relative weight of PFC on Striatum was increased from the default, so that it had an equal influence on Striatum as the PC layer. (e) Trajectory of response support over time when there is more influence of PFC than PC. The default relative weights of the PFC and PC projections were reversed, so that the PFC projection on Striatum had a higher relative weight (1.25) than the PC projection to Striatum (0.75). (f) Evolution of response support from each model manipulation (labeled in a-e) is represented using a tree structure, with thicker lines indicating more support for a particular branch. The four highlighted phases are as follows: 1. All responses are inhibited. 2. The responses that are correct on the low level (solid lines) have more support. The response correct on both levels has the most support. With less influence of PFC on Striatum (2a), support for the two responses correct on the low level are similarly supported. With more influence of PFC on Striatum (2b), the correct response has more support than all others. 3. Both responses correct on the top level (blue lines) get a late boost of support, while support for the top-level errors (red lines) goes to zero. 4. Support for the top correct/low error response decreases, and support for the response correct on both levels increases until the end of the trial.