Predictive Representations in Hippocampal and Prefrontal Hierarchies

Abstract

As we navigate the world, we use learned representations of relational structures to explore and to reach goals. Studies of how relational knowledge enables inference and planning are typically conducted in controlled small-scale settings. It remains unclear, however, how people use stored knowledge in continuously unfolding navigation (e.g., walking long distances in a city). We hypothesized that multiscale predictive representations guide naturalistic navigation in humans, and these scales are organized along posterior-anterior prefrontal and hippocampal hierarchies. We conducted model-based representational similarity analyses of neuroimaging data collected while male and female participants navigated realistically long paths in virtual reality. We tested the pattern similarity of each point, along each path, to a weighted sum of its successor points within predictive horizons of different scales. We found that anterior PFC showed the largest predictive horizons, posterior hippocampus the smallest, with the anterior hippocampus and orbitofrontal regions in between. Our findings offer novel insights into how cognitive maps support hierarchical planning at multiple scales.SIGNIFICANCE STATEMENT Whenever we navigate the world, we represent our journey at multiple horizons: from our immediate surroundings to our distal goal. How are such cognitive maps at different horizons simultaneously represented in the brain? Here, we applied a reinforcement learning-based analysis to neuroimaging data acquired while participants virtually navigated their hometown. We investigated neural patterns in the hippocampus and PFC, key cognitive map regions. We uncovered predictive representations with multiscale horizons in prefrontal and hippocampal gradients, with the longest predictive horizons in anterior PFC and the shortest in posterior hippocampus. These findings provide empirical support for the computational hypothesis that multiscale neural representations guide goal-directed navigation. This advances our understanding of hierarchical planning in everyday navigation of realistic distances.

Keywords: PFC; cognitive maps; hippocampus; navigation; predictive representations; successor representation.

Figure 1.

Schematic of the hypothesis, task conditions, and analytic methods. A, Multiple scales of representation along a navigated route are activated simultaneously. Longer predictive horizons correspond to longer-range planning and greater scales of navigational representations. B, Predictive representations in the hippocampus and PFC should proceed along a posterior-anterior gradient within the hippocampus and PFC. C, Participants used the same keys to navigate goal-directed routes and to follow the GPS dynamic arrow, but only goal-directed routes required goal-directed navigation. D, Analytic approach. The voxelwise pattern at each time point was correlated with the γ-weighted sum of all future states (for γ values of 0.1, 0.6, 0.8, and 0.9). With higher γ values, the weighted future states remain >0 further into the future. Not displayed: We also computed similarity for each TR to goal, and similarity of each TR to mean of future TRs (equally weighted) within a given horizon (e.g., 10 TRs). For details of the model-based analysis, see Materials and Methods.

Figure 2.

Descriptive statistics for navigated distances in goal-directed and GPS conditions. A, The goal-directed routes were rated as more familiar by participants than GPS routes. Goal-directed and GPS routes were matched in (B) ease of navigation and (C) speed of travel. D, GPS routes included more turns, on average, than goal-directed routes, but (E) the goal-directed routes tended to be longer than GPS routes.

Figure 3.

Similarity of each TR to mean of future TRs (equally weighted). Average correlation between each time point and the average of future 1/2/3/4/5 or 10 time points in (A) the goal-directed condition and (B) the GPS condition. C, Representational similarity for all ROIs across all temporal lags. More posterior regions cross zero at smaller horizons. The difference between the anterior and posterior hippocampus is less pronounced here than in subsequent analyses where neural representations were weighted by different discount factors. The aHPC and pHPC labels refer to anterior-most and posterior-most hippocampal segments, respectively, shown in A and B.

Figure 4.

Predictive similarity across predictive scales. Correlations between current time points and the ɣ-weighted sum of future states for different values of γ, in the four specified ROIs in the (A) goal-directed and (B) GPS conditions. ɣ = 0.1 only included 1 step (1 TR) away, ɣ = 0.6 reached ∼6 or 7 steps in the future, corresponding to ∼175 m, ɣ = 0.8, ∼14 steps or 350 m ahead.

Figure 5.

Linear mixed effects model predicting representational similarity (y axis) from path distance (x axis), ɣ, and ROI. Voxelwise patterns in different ROIs interacted differently with path distance: in the antPFC, routes with longer path distances were associated with greater representational similarity, whereas the opposite trend was present in the hippocampus (both aHPC and pHPC). Plot represents the model fit values and CIs. These reflect the relationships between the variables of interest after regressing out the effect of the number of TRs on each route and accounting for all other main effects and interactions.

Figure 6.

One-sample t tests for goal-directed and GPS condition. A, Voxels with significant representation of future states in the goal-directed and GPS conditions using a one-sample t test against zero. B, Voxels with representational similarity (correlation) values >0.06 for each value of ɣ. C, One-sample t tests with distance as covariate. The results look very similar to running a t test on goal-directed routes versus zero, and GPS routes versus zero. The mean distance, per participant, per condition, was included as a covariate. D, Discounted weights corresponding to different gammas were applied to each successor TR. The average distance covered in each TR was ∼25 m (24.8 m). Based on this, we computed approximate distances corresponding to predictive horizons for each discount parameter. The exact distances for each discount parameter differed across routes and participants depending on their speed. ɣ = 0.1 only included 1 step (1 TR) away, ɣ = 0.6 reached ∼7 steps in the future, corresponding to ∼175 m, ɣ = 0.8, ∼15 steps or 375 m, ɣ = 0.9 reached ∼32 steps or 800 m ahead.

Figure 7.

Increasing predictive similarity along posterior PFC to antPFC. In order to indicate which PFC regions displayed higher predictive similarity, we computed the slope of correlations for posterior PFC to antPFC slices for goal-directed and GPS conditions. We computed these slopes for 4 values of ɣ, corresponding to gradients of low to high scales. Each line indicates predictive similarity results from 1 of 19 participants.

Figure 8.

PFC hierarchy in the goal-directed and GPS conditions. Proportion of prefrontal BAs accounted for by the significant PFC voxels in searchlight analysis are shown. Results were driven from the one-sample t test results displayed in Figure 5A (not matched for distance). Color bars represent different discount values (ɣ) corresponding to different predictive horizons within each condition (blue: ɣ = 0.1; green: ɣ = 0.6; yellow: ɣ = 0.8; red: ɣ = 0.9).

Figure 9.

Predictive representations for goal-directed and GPS routes with matched distances. A, Distribution of distance-matched routes included in this analysis. B, Voxels with average correlation values of > 0.04. C, Significant voxels in goal-directed > GPS paired t test, thresholded at t value corresponding to 5% FPR. Colors represent predictive horizons corresponding to different discount parameters (blue: ɣ = 0.1; green: ɣ = 0.6; yellow: ɣ = 0.8; red: ɣ = 0.9).