Enhanced intersubject correlations during movie viewing correlate with successful episodic encoding

Abstract

While much has been learned regarding the neural substrates supporting episodic encoding using highly controlled experimental protocols, relatively little is known regarding the neural bases of episodic encoding of real-world events. In an effort to examine this issue, we measured fMRI activity while observers viewed a novel TV sitcom. Three weeks later, subsequent memory (SM) for the narrative content of movie events was assessed. We analyzed the encoding data for intersubject correlations (ISC) based on subjects' subsequent memory (ISC-SM) performance to identify brain regions whose BOLD response is significantly more correlated across subjects during portions of the movie that are successfully as compared to unsuccessfully encoded. These regions include the parahippocampal gyrus, superior temporal gyrus, anterior temporal poles, and the temporal-parietal junction. Further analyses reveal (1) that these correlated regions can display distinct activation profiles and (2) that the results seen with the ISC-SM analysis are complementary to more traditional linear models and allow analysis of complex time course data. Thus, the ISC-SM analysis extends traditional subsequent memory findings to a rich, dynamic and more ecologically valid situation.

Figure 1. Memory Test and Behavioral Performance

(A) Memory was assessed using an interactive computerized questionnaire. On each trial, a visual snapshot from the queried movie segment was presented with a memory question. Subjects were instructed to recall the answer to the question if they were confident they could. If they were not, a two-alternative force choice recognition phase ensued (left panel) during which they would choose their answer and rate their confidence (“highly confident,” “fairly confident,” or “guessing”; right panel). (B) Behavioral measures of performance. Subjects’ performance is presented as overall percent correct and percent correct for high- and low-confidence level. Dotted line represents the chance level of performance obtained by a group of subjects that completed the questionnaire without seeing the movie (Furman et al., 2007). Error bars are standard error of the mean (SEM). (Abbreviations: HCR = high-confidence recognition; LCR = low-confidence recognition).

Figure 2. ISC Based on SM

Example BOLD time course responses extracted from the same voxel of two representative subjects are shown. Probed events within each time course are color-coded according to subsequent memory performance with remembered in blue and missed in gray. For a given pair of subjects, memory performance and corresponding BOLD time courses are binned into three categories: both subjects remembered, both subjects missed, and subjects responded differently. Next, the ISC is calculated for each category separately, as detailed under Experimental Procedures, resulting in three separate correlation coefficients, or the ISC-SM. Finally, a statistical contrast is performed searching for voxels with significantly greater ISC during encoding of events later remembered compared to those missed.

Figure 3. Regions Showing Enhanced ISC during Events Later Successfully Remembered

Statistical maps displayed on sagittal (top) and coronal (bottom) sections of group mean anatomical brain depict brain areas with significantly enhanced ISC during movie segments that were subsequently remembered compared to segments that were subsequently missed. Voxels are colored according to the ISC R value for the both remembered condition. Moving from anterior to posterior, these regions include the right temporal pole (TP; [48, 11, –21]) (Talairach coordinates), bilateral anterior and posterior superior temporal gyrus (anterior STG: right [56, –12, –1]; left [–57, –12, 1]; posterior STG: right [52, –35, 4]; left [–57, –35, 4]), bilateral anterior parahippocampal cortex (aPHG; right ([25, –29, –18]; left [–29, –35, –17]), bilateral posterior parahippocampal gyrus (pPHG; right [23, –39, –14]; left [–27, –44, –14]), and bilateral temporal parietal junction (TPJ; right [51, –53, 9]; left [–55, –55, 12]). RH, LH denote the right and left hemisphere, respectively.

Figure 4. Comparison of ISC Computed with and without Subsequent Memory Constraints

Regions showing significant ISC during movie viewing are shown in red (same threshold as in Figure 3) and regions showing significant greater ISC during events later remembered versus missed are shown in yellow (replicated from Figure 3). The regions where these two maps overlap are shown in orange. The layout is identical to Figure 3. Visual cortical areas (y = –64 to –80) exhibited high ISC during movie watching (red), but this correlation was not enhanced for remembered compared to missed events. Other regions, such as pPHG(y= –37), STG (x = 49), and TPJ (x = 49) exhibited significant ISC for the entire time course, with enhanced correlation for the later remembered events (orange); while the ISC in the temporal pole (y = 7) and aPHG (y = –27) was significantly correlated only for events that were later remembered (yellow).

Figure 5. Trigger Averaged Responses in ISC-SM Regions of Interest

(A) Regions of interest (ROIs). The top five ROIs (red) were defined from the ISC-SM map (Figures 3 and S2) and are displayed on coronal slices from the group average anatomical brain: right medial prefrontal cortex (mPFC), right temporal pole, right temporal parietal junction (TPJ), right posterior parahippocampal gyrus (pPHG), and right anterior parahippocampal gyrus (aPHG). An additional control region was sampled from the vicinity of the calcarine sulcus (V1/V2). (B) Determination of significance using a nonparametric permutation method. Behavioral measures of each subject were permutated and then randomly reassigned. Plots present the distribution of the following statistic: the difference both remembered – both missed correlation coefficients, computed from the permutated “mock” behavioral data (for details see under Experimental Procedures). Red vertical line in each panel represents the value of the statistic when calculated using veridical behavioral measures. The red asterisk denotes significance of the nonparametric test. Note: the statistic calculated using veridical behavioral data is found at the positive tail of the distribution (>95% confidence level) in all regions except the control region (V1/V2). (C) Response amplitude averaged by the fact-availability. “Fact availability” was defined as the time point in the movie at which the probed content first becomes available. The BOLD time course from all subjects was Z normalized and averaged according to each subject's behavioral performance: remembered epochs (blue lines, include correct answers using recall and high-confidence recognition) or missed epochs (gray lines, include incorrect recognition answers). An X denotes a significant difference between the two conditions (two tailed t test). Black X represents significantly higher remembered trace, red X represents significantly higher missed trace. Yellow bars highlight time points where remembered and missed events show significant magnitude differences in the mPFC, right temporal pole, and TPJ, but not in the PHG. (D) Response amplitude averaged by episode boundary. All plots same as above, except in this case responses were trigger-averaged starting at the boundary between probed events. Green bars serve to indicate both the initial event-locked response seen in PHG but not in other regions as well as the later divergence of remembered and missed responses in PHG but not in other regions. (See Figure S4 for the same data using a deconvolution approach).

Figure 6. GLM Results Using a Simple Linear Regressor

(A) Regions showing a significant correlation with the derived linear regressor (GLM, with a random effects group map, n = 12). Data shown on sagittal views of the group-average anatomical brain. (B) For comparison, a contrast ISC-SM map is presented, as shown in Figure 3. Results of the derived linear model partially overlap with results of the ISC-SM analysis in the right temporal pole, STS, and TPJ (white circles) but do not overlap in the PHG (ISC-SM only, red circle) and IFG (linear fit only, yellow circle). The mPFC was significant in the uncorrected ISC-SM map (Figure S3) but did not survive the strict correction but again appears here in the linear fit (yellow circle).