Predicting the integration of overlapping memories by decoding mnemonic processing states during learning

Abstract

The hippocampal memory system is thought to alternate between two opposing processing states: encoding and retrieval. When present experience overlaps with past experience, this creates a potential tradeoff between encoding the present and retrieving the past. This tradeoff may be resolved by memory integration-that is, by forming a mnemonic representation that links present experience with overlapping past experience. Here, we used fMRI decoding analyses to predict when - and establish how - past and present experiences become integrated in memory. In an initial experiment, we alternately instructed subjects to adopt encoding, retrieval or integration states during overlapping learning. We then trained across-subject pattern classifiers to 'read out' the instructed processing states from fMRI activity patterns. We show that an integration state was clearly dissociable from encoding or retrieval states. Moreover, trial-by-trial fluctuations in decoded evidence for an integration state during learning reliably predicted behavioral expressions of successful memory integration. Strikingly, the decoding algorithm also successfully predicted specific instances of spontaneous memory integration in an entirely independent sample of subjects for whom processing state instructions were not administered. Finally, we show that medial prefrontal cortex and hippocampus differentially contribute to encoding, retrieval, and integration states: whereas hippocampus signals the tradeoff between encoding vs. retrieval states, medial prefrontal cortex actively represents past experience in relation to new learning.

Keywords: Episodic memory; Hippocampus; Integration; MVPA; Medial prefrontal cortex; Reinstatement.

Figure 1

Experimental design and behavioral results. (A) During acquisition rounds (8 total), subjects studied word-picture pairs (old pairs; 4s each). Pictures were drawn from three categories: faces, scenes, objects. (B) Each acquisition round was followed by a new learning round in which words from the immediately preceding acquisition round were paired with new pictures (new pairs, 2s each). After each word-picture pair disappeared, a shape cue (6s) instructed participants to: encode the new pair, retrieve the old pair, or integrate the old and new pairs. (C) After all of the acquisition/new learning rounds, subjects completed a surprise integration test. On each trial, a picture from the new learning rounds was presented and subjects attempted to remember the corresponding old picture (i.e., the picture that shared the same word cue). The integration test consisted of two steps: first participants indicated the category of the old picture (object, face, or scene; 4s maximum), and then subjects indicated the specific old picture from a set of 4 choices (all from the same visual category; 3s maximum). (D) Instructions during new learning (encode, retrieve, integrate) significantly influenced accuracy in selecting the specific picture (F_2,40 = 7.89, p = .001); there was a similar but non-significant pattern for the category-level decision (F_2,40 = 1.43, p = .25). Error bars correspond to standard error of the mean.

Figure 2

Decoding mnemonic processing states. (A) State decoding was performed using leave-one-subject-out cross-validation, in which classifiers were iteratively trained to decode the instruction received on each trial (retrieve vs. encode vs. integrate) using data from 20/21 subjects and then tested on each trial for the held-out subject. (B) Pairwise classification accuracy (for each pair of processing states) for the whole brain mask. (C) Three-way classification accuracy in sub-region masks. Dashed red line = performance of whole brain classifier. Error bars correspond to standard error of the mean. Notes: ** p < .005, one-tailed t-test; IFG = inferior frontal gyrus; MFG = middle frontal gyrus; SFG = superior frontal gyrus; MPFC = medial prefrontal cortex; OFC = orbitofrontal cortex; LTC = lateral temporal cortex; VTC = ventral temporal cortex; HIPP = hippocampus; IPL = inferior parietal lobule; SPL = superior parietal lobule; MPAR = medial parietal cortex; OCC = occipital cortex.

Figure 3

Predicting integration. (A) Classifier evidence for each processing state (i.e., retrieve evidence, encode evidence, integrate evidence) was used to predict behavioral performance on the post-scan integration test. Separate logistic regression analyses were performed for each subject and each instruction condition (to control for effects of instruction). Each bar represents the mean beta values from the regression analyses. Performance on the integration test was selectively predicted by classifier-derived evidence for an integration state during new learning. (B) A classifier was trained on data from the full sample of ‘instructed’ subjects and applied to data from a separate set of ‘uninstructed’ subjects. Trial-level classifier evidence from the uninstructed subjects was then used to predict performance on the integration test (as in A). Again, evidence for an integration state during new learning predicted performance on the integration test. (C) The across-subject correlation between percentage of new learning trials labeled by the classifier as ‘integrate’ and mean category-level accuracy on the subsequent integration test was marginally significant [data are collapsed across instructed (black) and uninstructed (green) samples, but separate trend lines are shown for each group for comparison]. (D) Same as (C) except that integration test performance (y-axis) reflects mean category + item accuracy. (E) Rank ordered category + item accuracy for individual subjects [combining across instructed (black) and uninstructed (green) samples]. Among the small sub-group of subjects with accuracy above 30% (n = 5), there was a robust trial-level relationship between classifier-derived evidence for an integration state during new learning and category + item level accuracy on the subsequent integration test. Notes: * p < .05, ** p < .005; a two-tailed t-test was used for (A), but a one-tailed t-test was used for (B) given that the analysis was a replication with a clear directional prediction.

Figure 4

Decoding reactivation. (A) Pattern classifiers were trained to discriminate visual category information (face vs. scene vs. object) using data from the acquisition phase. The classifiers were then tested on each trial in the new learning phase to measure the strength of evidence for the category of the old picture (as well as for the baseline and new picture categories); evidence for the baseline category was subtracted from evidence for the old category to obtain a measure of reactivation. (B) Reactivation in the whole brain mask as a function of instruction condition. (C) Reactivation in twelve sub-regions of the whole brain mask, separately for the instructed and uninstructed subjects. Error bars correspond to standard error of the mean. Notes: ** p < .005, one-tailed t-test; IFG = inferior frontal gyrus; MFG = middle frontal gyrus; SFG = superior frontal gyrus; MPFC = medial prefrontal cortex; OFC = orbitofrontal cortex; LTC = lateral temporal cortex; VTC = ventral temporal cortex; HIPP = hippocampus; IPL = inferior parietal lobule; SPL = superior parietal lobule; MPAR = medial parietal cortex; OCC = occipital cortex.

Figure 5

Process decoding in sub-regions of interest. (A) Pairwise classification accuracy for each pair of instruction conditions across sub-regions. (B) Trial-by-trial fluctuations in classifier evidence for each processing state were used to predict category-level behavioral performance on the post-scan integration test using logistic regression analyses (as in Figure 3A) for each of the sub-regions. Each bar represents the mean beta values from separate regressions for each form of classifier evidence (retrieve, encode, integrate) and each sub-region. Notes: ** p < .01; * p < .05; ~ p < .1. One-tailed t-tests were used for (A) given that classifier accuracy was compared to chance, but two-tailed tests were used in (B). Performance on the integration test was positively predicted by classifier-derived integrate evidence in MPFC (marginally significant) and VTC. Integration test performance was negatively predicted by retrieve evidence in MFPC and encode evidence in VTC.

Figure 6

Reactivation in sub-regions of interest. (A) Pattern classifiers were trained to discriminate visual category information (face vs. scene vs. object) using data from the acquisition phase and were then tested on each trial in the new learning phase. Classifier evidence for the baseline category (i.e., the category to which neither the old nor new picture belonged) was subtracted from evidence for the old category to obtain a measure of reactivation. Across the sub-regions, reactivation was greater for integrate and retrieve trials than encode trials. HIPP was characterized by relatively weaker reactivation on integrate than retrieve trials, which contrasted with MPFC. (B) Trial-by-trial fluctuations in reactivation strength during new learning were related to category-level accuracy on the subsequent integration test using subject-specific logistic regression analyses (data from instructed and uninstructed subjects were combined). Individual bars reflect mean beta values from these regression analyses, separately for each sub-region. Reactivation in VTC positively predicted subsequent performance on the integration test. (C) Individual differences in mean reactivation during the new learning phase were correlated with category + item accuracy on the subsequent integration test, separately for each sub-region. Significant across-subject correlations were observed in MPFC and VTC. [Notes: ** p < .01; * p < .05; ~ p < .1. One-tailed t-tests were used for (A) given that reactivation was compared to baseline, but two-tailed tests were used in (B). the correlations combined the instructed (black) and uninstructed (green) samples, but separate trend lines are shown for each group for comparison].