The Importance of Semantic Network Brain Regions in Integrating Prior Knowledge with an Ongoing Dialogue

Abstract

To understand a dialogue, we need to know the topics that are being discussed. This enables us to integrate our knowledge of what was said previously to interpret the current dialogue. This study involved a large-scale behavioral experiment conducted online and a separate fMRI experiment, both testing human participants. In both, we selectively manipulated knowledge about the narrative content of dialogues presented in short videos. The clips were scenes from situation comedies that were split into two parts. The speech in the part 1 clips could either be presented normally or spectrally rotated to render it unintelligible. The part 2 clips that concluded the scenes were always presented normally. The behavioral experiment showed that knowledge of the preceding narrative boosted memory for the part 2 clips as well as increased the intersubject semantic similarity of recalled descriptions of the dialogues. The fMRI experiment replicated the finding that prior knowledge improved memory for the conclusions of the dialogues. Furthermore, prior knowledge strengthened temporal intersubject correlations in brain regions including the left angular gyrus and inferior frontal gyrus. Together, these findings show that (1) prior knowledge constrains the interpretation of a dialogue to be more similar across individuals; and (2), consistent with this, the activation of brain regions involved in semantic control processing is also more similar between individuals who share the same prior knowledge. Processing in these regions likely supports the activation and integration of prior knowledge, which helps people to better understand and remember dialogues as they unfold.

Keywords: fMRI; memory; prior knowledge; semantic.

Figure 1.

Schematic of study design. Participants viewed clips from unfamiliar TV sitcoms that were divided into two parts. Top row shows the example order of stimuli (e.g., S1, show 1; S2, show 2). Blue designates LC pairs of part 1 and part 2 clips; orange designates pairs of HC clips. Participants viewed a set of 5 part 1 clips followed by a set of 5 corresponding part 2 clips presented in random order. Ten of the part 1 clips had comprehensible speech, whereas the other 10 had unintelligible speech, created by spectrally rotating the audio. Clips were counterbalanced across participants in a within-subjects design (see middle and bottom rows). All main analyses were on the part 2 clips, which were identical across conditions and differed only in whether participants had knowledge of the preceding (part 1) dialogue (HC condition) or did not (LC condition).

Figure 2.

Subjective memory ratings were better for the HC videos. Left column shows results from subjective measures in experiment 1, and right column shows subjective measures in experiment 2. Error bars represent standard error of the mean. **p < 0.01, ***p < 0.001.

Figure 3.

Participants remembered the HC videos better. We show that free-recall accuracy was higher for the HC videos compared with the LC videos. Participants in experiment 2 showed better memory, tested with open-ended memory questions, for the HC videos compared with the LC videos. Error bars represent standard error of the mean. **p < 0.01; ***p < 0.001.

Figure 4.

Semantic consistency is higher among recall of HC videos. A, A PCA projection in 2 dimensions of the semantic vectors that represent all participants’ correct free-recall responses for a single video. Semantic consistency for each video was calculated within each condition. The larger center dots are the condition average semantic patterns for the video based on the free-recall responses. Lines toward the center are for illustrative purposes. B, We calculated the average spread of scores for each video under each condition. One data point in the bar graph represented the average spread of responses across participants watching a video in one condition (e.g., average spread in orange points in A). Semantic consistency was on average higher for the HC videos. Error bars represent 95% confidence intervals. ***p < 0.001.

Figure 5.

Intersubject correlations. A, The weight matrix (General ISC) tests for video-specific time course similarity across participants. Each cell represents the correlation between subjects’ time course for a particular video with the average time course of all remaining participants for a particular video. The diagonal represents correlations between time courses for the same videos. The off-diagonal represents temporal correlations between mismatching videos within the same run. B, Brain map from video-specific analysis, which shows extended synchronization across the brain for people watching the same videos. C, Weight matrix that tests for the time course similarity across the same videos, depending on their prior knowledge. D, Brain map showing how time course synchronicity was modulated by prior knowledge. Both brain maps show clusters significant at FWE p < 0.05 after permutation testing.