Frontal cortex function as derived from hierarchical predictive coding

Abstract

The frontal lobes are essential for human volition and goal-directed behavior, yet their function remains unclear. While various models have highlighted working memory, reinforcement learning, and cognitive control as key functions, a single framework for interpreting the range of effects observed in prefrontal cortex has yet to emerge. Here we show that a simple computational motif based on predictive coding can be stacked hierarchically to learn and perform arbitrarily complex goal-directed behavior. The resulting Hierarchical Error Representation (HER) model simulates a wide array of findings from fMRI, ERP, single-units, and neuropsychological studies of both lateral and medial prefrontal cortex. By reconceptualizing lateral prefrontal activity as anticipating prediction errors, the HER model provides a novel unifying account of prefrontal cortex function with broad implications for understanding the frontal cortex across multiple levels of description, from the level of single neurons to behavior.

Figure 1

Predictive Coding in Prefrontal Cortex. (A) In the HER model, information is passed to hierarchical levels through bottom-up and top-down pathways. In the bottom-up paths (top), regions in mPFC compute an error signal as the discrepancy between the expected and actual output of inferior hierarchical levels. Error signals generated by mPFC train error predictions in lateral PFC which are associated with task stimuli that reliably precede them. Following training, learned representations of error predictions are elicited by task stimuli and actively maintained in dlPFC for as long as they have predictive value. In the top-down pathway (bottom), error predictions are passed from superior hierarchical levels in order to successively modulate predictions made at inferior levels. (B) The organization of the HER model is similar to formulations of predictive coding and free energy previously used to explain results from early sensory processing areas and hypothesized to extend into the frontal lobes. Figure reprinted with permission from Friston. (C) A detailed circuit diagram of the HER model shows bottom-up (red and green) and top-down (violet) pathways, as well as the working memory gating mechanism that allows information to be maintained over extended durations. The connections match known neuroanatomy^,.

Figure 2

Information encoding in dlPFC. Simulated data is enclosed in double-bordered boxes throughout the manuscript. As the information content of a context cue increases, calculated in Bits (X axis), activity across hierarchically organized regions of dlPFC increases. The strength of error predictions maintained in dlPFC is proportional to information content: the more informative a cue is, the larger a reported error will be without the information supplied by that cue. (A) The HER model captures effects of information related both to the nature of task-relevant stimuli (x axes) as well as responses that may be required (y axes). The HER thus provides a complementary account to the Information Cascade model of PFC. (B) In the Task Condition of Koechlin et al., activity across dlPFC is observed to increase with the information content of a contextual cue. However, here activity in caudal dlPFC (panel B, middle) shows an additional increase when subjects must occasionally switch between two tasks (vowel/consonant, upper/lower case identification). (C) This additional increase related to task switching is accounted for as transient increases in activity in the HER model when the nature of the task changes (middle row).

Figure 3

Distributed Representations in PFC. Separate units in the HER model represent components of a hierarchically-elaborated, multi-dimensional error prediction, suggesting how cognitive tasks may be represented neurally. (A) Left: MVPA on error prediction representations maintained by the model while performing the 1-2AX CPT are consistent with human data showing that caudal regions of lPFC code for potential target sequences regardless of higher-order context, while more rostral regions encode more abstract context variables. Right: Human MVPA results, reprinted by permission from Nee & Brown. Classification results of model representations are naturally more robust than pattern analysis of fMRI data since it is possible to record the activation of units in the model with perfect fidelity, while BOLD signals are subject to noise. Nevertheless, classification accuracy for model representations was significantly correlated with classification accuracy for human data at both hierarchical level 2/mid-DLPFC (r = 0.64, p = 0.0074) and level 1/dorsal premotor cortex (r = 0.91, p < 0.001). (B) Units in level 1 of the HER model (left) show activity related to match suppression and enhancement while performing a delayed match-to-sample task. Prior to observing a target stimulus, activity in these units reflects the equal probability of observing a match or non-match cue. Following the presentation of the target stimulus, the activity of units predicting the occurrence of a match is enhanced, while the activity for non-match-predicting units is suppressed, similar to data recorded from monkey lPFC (right). The HER model further predicts the existence of units showing effects of mismatch enhancement and suppression. Reprinted by permission from Miller et al..

Figure 4

Mixed Selectivity in the HER model. Units from level 2 of the HER model show complex patterns in response to stimuli. The responses of units 1–4 show interactions between a cued rule and a sample stimulus presented during a DMTS task, with some units preferentially responding to, e.g., combinations of ‘Same’ rules and ‘Picture 1’. Additional units in the model (5 & 6) respond solely to a preferred rule: ‘Same’ (Unit 6, panel B) or ‘Different’ (Unit 5, panel A). The combination of rule-specific and ruleXcue interaction units replicates similar findings in primate LPFC.

Figure 5

Connecting representations to behavior. Behavior of the HER model (top) with learning selectively enabled at zero (left), one (center), and all (right) hierarchical levels. Each point represents a single trial. The model’s estimate of the probabilities of three possible categories matches the behavior of three groups of human subjects with varying information sampling strategies (bottom) during a ternary probability estimation task. The HER model thus provides an account of how task representation at the level of single units contributes to behavior. Reprinted by permission from.

Figure 6

Interactions of mPFC and dlPFC, Simulations 6 and 7. The HER model suggests how mPFC and dlPFC may cooperate to minimize prediction error through passing error and error prediction information through hierarchical levels. (A) Simulation 6. Increased activity in parallel hierarchical regions in the HER model, associated with mPFC and dlPFC, is associated with errors (mPFC) and updates of error predictions (dlPFC) at different levels of abstraction, from concrete (level 1, stimulus switch) to abstract (level 2, response switch; level 3, context switch). (B) Simulation 7. Modulation of mPFC by error predictions maintained in dlPFC is critical for contextualizing predictions regarding the likely outcome of actions. In a delayed-match-to-sample task, the HER model correctly captures the elimination of the ERN following correct trials due to the maintenance of information regarding the sample cue. However, when the model is lesioned such that information normally maintained in dlPFC is no longer available to mPFC, the model produces an ERN to correct and error trials alike.