Generative replay underlies compositional inference in the hippocampal-prefrontal circuit

Abstract

Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.

Keywords: cognitive maps; compositional inference; flexible reasoning; neural replay; prefrontal-hippocampal circuit.

Graphical abstract

Figure 1

Paradigm and behavioral training (A) On 2 consecutive days, subjects were trained on nine basic building blocks, which could be flexibly combined by placing one building block on top of (below) or beside (left or right) another building block. (B) The complexity of the target silhouettes increased gradually, and subjects achieved overall high performance both in the actual construction (left) and determination of present building blocks under time pressure (right). Shaded colored areas reflect standard errors. See also Figure S1.

Figure S1

Behavioral effects, related to Figure 1 (A) We included an implicit hierarchical structure in the task, such that large silhouettes could often be decomposed into hierarchical building blocks. These hierarchical building blocks were never introduced explicitly but allowed for a more efficient construction of larger objects once learned. (B) Subjects displayed a preference for such “hierarchical chunking,” such that on the second training day they used a hierarchical building block configuration to construct larger silhouettes more often than predicted by chance. (C) Preferences for hierarchical chunking for the individual hierarchical building blocks. (D) At the end of the experiment, subjects completed a behavioral questionnaire to indicate similarity judgments between silhouettes. We found these similarity judgments were influenced by visual similarity, namely shape (pixel) and size overlap, and also by “construction similarity,” namely by the overlap of (basic/hierarchical) building blocks (BBs) across (small/large) silhouettes.

Figure S2

Stimulus properties, related to STAR Methods Average size for non-hierarchical (top row) and hierarchical (bottom row) compounds built by placing one (basic or hierarchical) building block on top of (left column) or beside (right column) another (basic or hierarchical) building block.

Figure 2

Neural effects of visual processing (A) In the fMRI-scanner, subjects saw a silhouette for a short period of time and were instructed to infer a plan for the construction of that silhouette. Sometimes trials were followed by a catch trial, in which subjects had to indicate whether blocks were part of the construction of the previous silhouette. (B) In the scanner, subjects received (known) basic building blocks (first row), hierarchical building blocks (second row), or novel and previously unseen compounds as construction trials. The novel compound silhouettes were either built with two basic building blocks on top of each other (third and fourth row) or beside each other (fifth and sixth row) or with two hierarchical building blocks on top (seventh and eighth) or beside (ninth and tenth) each other. (C) We found that activity in the lateral occipital, superior parietal, and precentral gyrus covaried with the number of elements in a compound, providing an approximation to construction difficulty (left). We also found effects for (absolute) changes in the number of elements between consecutive silhouettes in the lateral occipital cortex (middle). We did not detect any significant effects for differences in visual shape (pixels) but detected effects in superior parietal and frontal cortex reflecting differences in size between the individual silhouettes.

Figure 3

Construction inference is relational and compositional (A) We designed an analysis to test for generalizable representations of individual building blocks in specific relational positions by performing algebraic operations with neural representations for different silhouettes. For given building blocks WXYZ, the silhouette algebra predicts that $\frac{W}{X}-\frac{Y}{X}+\frac{Y}{Z}=\frac{W}{Z}$ (note these are spatial relations of blocks, not fractions). Under a conjunctive representation, the algebraic term on the left should be predictive of the actual silhouette with building block W on top of building block Z (target) but not of a silhouette with building block Z on top of building block W (reference). (B) Left: we tested in which brain regions such algebraic terms are predictive of target silhouettes but not reference silhouettes, using RSA, where we assessed whether the distance (defined as 1-correlation between activity patterns) between algebraic terms and target silhouettes is smaller than between algebraic terms and reference silhouettes (see STAR Methods for details). Right: we found significant effects in mPFC and the anterior hippocampus, extending into the entorhinal cortex, suggestive of a conjunctive representation of building blocks in specific relational positions. (C) Using repetition suppression, we probed the neural representations encoding for individual building blocks in a given construction problem, using an approach we reported in previous work when subjects had to imagine and evaluate novel food items. In regions encoding such representations, we expect higher suppression for transitions between silhouettes that share building blocks than transitions of silhouettes that use different building blocks. As predicted, we found the strongest suppression effects in the medial prefrontal cortex (red), highly overlapping with representations underlying the construction and evaluation of novel food items reported earlier (green, Barron et al. ; Figure 2C).

Figure S3

Stimulus properties silhouette algebra, related to STAR Methods All non-hierarchical (left) and hierarchical (right) silhouette algebra trials.

Figure 4

MEG task (A) The task consisted of an inference and probe phase. During inference, subjects were presented with a silhouette and had to infer its relational composition. During probe, subjects were presented with two building blocks and were asked to indicate the relation between these two building blocks in the previous silhouette, if any. (B) Subjects’ performance on the task improved over time. (C) The MEG experiment started with a functional localizer, where subjects observed individual building blocks with different textures (wood, concrete, steel, or bricks) on the screen. Intermittently, they received a probe question. The functional localizer was followed by a rest session, followed by three task sessions. The task was identical to training, except that we included an additional probe time window in which subjects were asked to infer the relation between two building blocks but could not yet indicate a response. The three task sessions were followed by another rest, followed by another three task sessions and a final rest session. (D) Subjects’ performance again improved over time, such that the proportion of correct responses increased, and reaction times decreased, with ongoing task experience. (E) In the MEG experiment, one building block was always present in every silhouette (stable, highlighted in red for an example stimulus set, see Figure S4 for all used stimuli), whereas two out of the remaining three had to be inferred (present) and one building block was absent. Shaded colored areas reflect standard errors.

Figure 5

Conjunctive representations akin to the silhouette algebra from Figure 3B over time using RSA (A) Left: we defined a theoretical similarity reflecting the overlap of building blocks in specific relations across silhouettes, and we tested whether this similarity predicts empirical similarities of MEG sensor patterns across trials and time points. Right: we found a significant conjunctive representation, reflecting representations of silhouettes according to their constituent building blocks in specific relations, during a confined time window of 200–1,000 ms in the inference phase (significance assessed using a non-parametric permutation test, see STAR Methods for details). (B) We also found effects for shape (pixel) and size representational overlap during a similar time window during inference but with a slightly earlier onset. Note that the purple line in (A) and (B) are the same. Shaded colored areas reflect standard errors, and dotted lines reflect the statistical threshold obtained from a sign flip permutation test.

Figure 6

Neural replay in construction inference (A) We found peak decoding accuracy for building blocks in the localizer at 200 ms (left and middle) and high-class identifiability for the different building blocks for the classifiers trained at 200 ms (right). (B) In every silhouette, one building block was stable across silhouettes, two additional building blocks were present, and one building block was absent. This allowed us to define different types of sequences to (green) and from (red) the stable building block as well as between the present (purple) and absent (cyan) building blocks. (C) We investigated effects of neural replay for sequences starting either from the stable or the present building blocks. We found a short (non-significant) predominance of sequences starting from the stable building block for very early lags, followed by a predominance of sequences starting from the present building blocks at later lags with pronounced peaks at 60 and 170 ms. Shaded colored areas reflect standard errors. See also Figure S4.

Figure S4

Types of building blocks in MEG task, related to Figures 6 and 7 and STAR Methods (A) Different stimulus sets used in the MEG task. Subjects were randomly assigned to one of these stimulus sets. (B) In half of the trials, the stable building block was not in the middle of the silhouette. (C) In the other half of trials, the stable building block was in the middle, such that there was no present to present building block connection.

Figure S5

Sensor distribution, related to STAR Methods Sensor distribution of classifier weights for all (left) and individual (right) building blocks, trained on functional localizer data.

Figure 7

Generative neural replay underlying hypothesis testing over timescales of online computation (A) We investigated the difference between sequences starting either from the stable or the present building blocks for different time intervals of the inference period, and we found a brief early predominance of replay starting from the stable building block followed by a predominance of replay starting from the present building blocks (260–1,660 ms) during inference. (B) We assessed the individual contributions of the different types of neural replay to these differences and found an unspecific predominance of sequences from the present (180–1,620 ms), distant present (the present block that is unconnected to stable, 170–1,680 ms), and absent (190–1,580 ms) building blocks to the stable building block early during inference, as well as a specific effect from present to the stable building block late (1,590–3,500 ms) in inference shortly before subjects entered the decision phase of the task. (C) We found a selective predominance of replay between present building blocks over replay between present and absent building blocks in a time window between 260 and 1,650 ms. (D) We also tested for length-3 replay effects using this sliding window approach. This implies testing whether a specific transition between two building blocks predicts the transition to a third building block, while controlling for all possible length-2 and alternative length-3 transitions. Using this approach, we found significant effects for length-3 replay reflecting sequences from [present to present] to stable (100–1,650 ms), [distant present to present] to stable (110–1,800), and [present to distant present] to stable (130–1,590 and 2,080–3,420 ms). Shaded colored areas reflect standard errors. See also Figures S4, S6, and S7.

Figure S6

Discrete effects generative replay, related to Figure 7 We investigated the difference between sequences starting either from the stable or the present building blocks for different time intervals of the inference period, and we found an early predominance of replay starting from the stable building block (0–1,000 ms) followed by a predominance of replay starting from the present building blocks (500–1,500 ms) during inference (left). Assessing the individual contributions of the different types of neural replay to these differences, we found a marked decrease of sequences toward the stable building block early during inference (0–1,000 ms) followed by a predominance of sequences starting from the present building blocks (500–1,500 ms). We also found a specific predominance of sequences from present to the stable building block during intervals at the end of the inference period (2,000–3,000 and 2,500–3,500 ms) before subjects entered the decision phase of the task.

Figure S7

Classifier reactivation and replay effects during the probe phase, related to Figure 7 (A) We investigated the time course of the classifier reactivations for the stable, present, and absent building blocks averaged across trials. All reactivations peak shortly after stimulus (silhouette) onset, with the fully predictable stable building block representation peaking earlier. Overlaid are the time windows of the significant replay effects from Figures 6E and 6F (orange, significant effects for sequences from candidate building blocks to stable building block; dark green, significant difference between sequences between present and between absent and present building blocks; light green, significant effects for sequences from present to stable). (B) Left: while displaying a similar tendency as during the inference phase, we did not find significant replay effects for individual sequences analogous to Figure 7B during the probe phase. Right: we also did not detect replay for more task relevant information during the probe phase, such as the connection from the probe block in the upper left corner of the screen to the probe block in the middle or vice versa (right).