Stimulus set meaningfulness and neurophysiological differentiation: a functional magnetic resonance imaging study

Abstract

A meaningful set of stimuli, such as a sequence of frames from a movie, triggers a set of different experiences. By contrast, a meaningless set of stimuli, such as a sequence of 'TV noise' frames, triggers always the same experience--of seeing 'TV noise'--even though the stimuli themselves are as different from each other as the movie frames. We reasoned that the differentiation of cortical responses underlying the subject's experiences, as measured by Lempel-Ziv complexity (incompressibility) of functional MRI images, should reflect the overall meaningfulness of a set of stimuli for the subject, rather than differences among the stimuli. We tested this hypothesis by quantifying the differentiation of brain activity patterns in response to a movie sequence, to the same movie scrambled in time, and to 'TV noise', where the pixels from each movie frame were scrambled in space. While overall cortical activation was strong and widespread in all conditions, the differentiation (Lempel-Ziv complexity) of brain activation patterns was correlated with the meaningfulness of the stimulus set, being highest in the movie condition, intermediate in the scrambled movie condition, and minimal for 'TV noise'. Stimulus set meaningfulness was also associated with higher information integration among cortical regions. These results suggest that the differentiation of neural responses can be used to assess the meaningfulness of a given set of stimuli for a given subject, without the need to identify the features and categories that are relevant to the subject, nor the precise location of selective neural responses.

Fig 1. Experimental paradigm.

Left panel: Schematic representation of a human brain with 3 representative voxels whose fMRI BOLD activity will be measured in a block design analysis and in a differentiation analysis. The red voxel is known to respond to faces, the orange voxel to places, and the green voxel has unknown selectivity. The colored traces in the right side panels represent the expected BOLD signal of these representative voxels during the fMRI experiments. Right, Top Panel: in the block design paradigm, 20 seconds sequences of movie, scrambled movie, or ‘TV noise’ are presented in alternation with a black screen baseline. The block design analysis is expected to reveal significant increases in the mean activity of the three pictured voxels for each of the three stimulus sequences compared to the black screen baseline. Bottom Panel: in the differentiation analysis paradigm, a 4 min sequence of movie, scrambled movie or ‘TV noise’ is presented to the subjects, each sequence repeated 30 times across different scanning sessions (only 3 of these repetitions are depicted, corresponding to 3 BOLD activity traces per voxel). In all three conditions, we expect an overall activation with respect to the black screen baseline similar to that in the block design paradigm. However, unlike the block design analysis, the differentiation analysis focuses on systematic time-locked increases or decreases in activity with respect to: i) each voxel’s the black screen baseline (dashed black line); ii) each voxel’s mean activity during the 4 min sequence. Movie sequence: for each voxel, we expect systematic time-locked increases and decreases of activity across the session (neurophysiological differentiation over time); moreover, we expect different voxels to show different patterns of systematic activations/deactivations in response to different movie frames (neurophysiological differentiation over space). Altogether, high neurophysiological differentiation in space and time (many different spatio-temporal patterns) is expected to go along high phenomenological differentiation (many different experiences). Scrambled movie sequence: we expect intermediate levels of neurophysiological differentiation, corresponding to intermediate levels of phenomenological differentiation. TV noise sequence: we expect no or minimal systematic time-locked incease or decreases in activity. Low neurophysiological differentiation (a single, unchanging pattern of activation/deactivation) corresponds to low phenomenological differentiation (a single, unchanging experience of ‘TV noise’). Spontaneous fluctuations in BOLD activity from scan to scan are also expected in the ‘TV noise’ session, but they will not be time locked to specific ‘TV noise’ frames, which cortical regions treat as equivalent. For copyright reasons, all movie pictures were replaced in Figures by numbered blank frames representing their order of appearance in the movie.

Fig 2. Block design results for movie, scrambled movie and ‘TV noise’ sequences.

Figure displays overall brain activation for movie, scrambled movie and ‘TV noise’ sequences as measured in the block design paradigm. Results shown in a representative subject for T contrasts comparing movie, scrambled movie and ‘TV noise’ to a black screen baseline, thresholded at whole brain FWE corrected p<0.05.

Fig 3. Lempel-Ziv complexity of brain activity correlates with stimulus set meaningfulness—comparison to a black screen baseline.

Results shown for a representative subject (same subject as for Fig 2). For each condition, rectangles in the left column show exemplar pixels at the center of the screen of each frame, illustrating that all stimulus set present a high level of physical differentiation over time. In contrast, brain activity patterns over time are highly differentiated in the movie condition, intermediately differentiated in the scrambled movie condition, and very similar to one another in the ‘TV noise’ condition. Brain maps are here expressed in terms of significant changes in activity as compared to a black screen baseline (F-test, thresholded at whole brain FWE corrected p<0.05 for each frame). Top panel displays binarized spatio-temporal activation/deactivation matrices obtained for the 3 conditions after statistical thresholding was applied: a value of 1 was assigned to above threshold voxels for each scan, and a value of zero to voxels below threshold. For display purposes, binarized activation matrices are displayed only for the voxels that show at least once a significant activation in the movie (data dimension reduction from 94000 to ~7000 voxels). Lempel-Ziv complexity was computed at the whole brain activation matrix encompassing 94000 voxels in each condition.

Fig 4. Lempel-Ziv complexity group values—comparison to a black screen baseline.

Left panel: overall activations/deactivations (F test) group values. Middle panel: Lempel-Ziv complexity values for activations only (positive T test) Right panel: Lempel-Ziv complexity values for deactivations only (negative T test). For display purposes, each subject’s Lempel-Ziv complexity was normalized by its individual maximum value across all conditions. Bar graphs show group mean and standard error of the mean in each condition.

Fig 5. Lempel-Ziv complexity of brain activity correlates with stimulus set meaningfulness—comparison to the sequence mean.

Figure displays differentiated brain activity patterns for movie, scrambled movie, and ‘TV noise’ stimulus sequences for the representative subject also used in Figs 2–3. Brain activity patterns over time are highly differentiated in the movie condition, intermediate in the scrambled movie condition, and very low in the ‘TV noise’ condition. Brain maps are here expressed in terms of significant changes in activity as compared to the within-session mean (F-test) thresholded at whole brain FWE corrected p<0.05 for each frame. Top panel displays binarized spatio-temporal activation/deactivation matrices obtained for the 3 conditions after statistical thresholding was applied—where a value of 1 was assigned to above threshold voxels for each scan, and a value of zero to voxels below threshold. For display purposes, binarized activation matrices are displayed only for the voxels that show at least once a significant activation in the movie (data dimension reduction from 94000 to ~900 voxels). Lempel-Ziv complexity was computed at the whole brain activation matrix encompassing 94000 voxels in each condition.

Fig 6. Group results for Lempel-ziv complexity analyses—comparison to the sequence mean.

Lempel-Ziv complexity values for movie, scrambled movie, and 'TV noise'. Left panel: overall activations/deactivations (F test) group values. Middle panel: complexity values for activations only (positive T test) Right panel: complexity values for deactivations only (negative T test). For display purposes, each subject’s Lempel-Ziv complexity was normalized by its individual maximum value across all conditions. Bar graphs show group mean and standard error of the mean in each condition.

Fig 7. Group results for Integrated information Φ* analyses.

Integrated information Φ* results for the movie, scrambled movie, and ‘TV noise’ conditions. Colors of bars represent conditions (dark gray: movie, medium gray: scrambled movie, light gray: TV noise). Upper left panel: the group-mean of Φ* calculated by using time lag of 4 second and ROI set 1. Error bar represents standard error of the mean for each measure in each condition. Asterisks indicate significant differences of the group means (p<0.05, corrected). Upper right panel: the group-mean of Φ* calculated by using time lag of 4 second and ROI set 2. Lower panel: Φ* calculated in our representative subject with ROI set 1, showing robust results across different time lags (1–10 seconds). In this panel, error bar indicates standard deviation of the mean for 1000 sets of 80 ROIs (see Methods).

Fig 8. Group results for Neural Complexity analyses.

Neural Complexity results for the movie, scrambled movie, and ‘TV noise’ conditions. Colors of bars represent conditions (dark gray: movie, medium gray: scrambled movie, light gray: TV noise). Left panel: group means and standard deviation of the mean for Neural Complexity calculated on ROI set 1. Right panel: group means and standard deviation of the mean for Neural Complexity calculated on ROI set 2. Asterisks indicate significant differences of the group means (p<0.05, corrected).