How do neural processes give rise to cognition? Simultaneously predicting brain and behavior with a dynamic model of visual working memory

Abstract

There is consensus that activation within distributed functional brain networks underlies human thought. The impact of this consensus is limited, however, by a gap that exists between data-driven correlational analyses that specify where functional brain activity is localized using functional magnetic resonance imaging (fMRI), and neural process accounts that specify how neural activity unfolds through time to give rise to behavior. Here, we show how an integrative cognitive neuroscience approach may bridge this gap. In an exemplary study of visual working memory, we use multilevel Bayesian statistics to demonstrate that a neural dynamic model simultaneously explains behavioral data and predicts localized patterns of brain activity, outperforming standard analytic approaches to fMRI. The model explains performance on both correct trials and incorrect trials where errors in change detection emerge from neural fluctuations amplified by neural interaction. Critically, predictions of the model run counter to cognitive theories of the origin of errors in change detection. Results reveal neural patterns predicted by the model within regions of the dorsal attention network that have been the focus of much debate. The model-based analysis suggests that key areas in the dorsal attention network such as the intraparietal sulcus play a central role in change detection rather than working memory maintenance, counter to previous interpretations of fMRI studies. More generally, the integrative cognitive neuroscience approach used here establishes a framework for directly testing theories of cognitive and brain function using the combined power of behavioral and fMRI data. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Figure 1.

Illustration of activation dynamics. (A and B) The phase-space and activation over time of a neuron with linear dynamics. The purple line in panel A corresponds to the period of time in panel B during which activation is boosted by an input, the red line in panel A corresponds to the other time points. (C and D) The phase-space and activation over time of a neuron with nonlinear dynamics created through the addition of self-excitation (note the curves in phase-space around the activation value of 0). When the neuron is boosted by an input in panel D, self-excitation creates a nonlinearity that pulls activation fluctuations push activation back below 0 and self-excitation is disengaged. (E and F) Corresponding activation profiles for these two different systems in a field of interactive neurons. Note the correspondence in profiles between B–E and D–F. Solid arrows in A and C indicate the location of attractors and the dashed arrow indicatest the location of a repeller. See the online article for the color version of this figure.

Figure 2.

Model architecture. Excitatory connections are indicated by lines with pointed end and inhibitory connections are illustrated with lines with balled end. Connections with parallel lines (i.e., between “Different” and contrast field [CF] and between “Same” and working memory [WM]) are engaged when the Gate node is activated. Connections with perpendicular lines (i.e., from CF to WM) are turned off when the Gate node is activated. See the online article for the color version of this figure.

Figure 3.

Model dynamics. (A) Activation of the model architecture on a set-Size 3 trial. (B) Activation of the decision nodes over the course the trial. (C and D) Time-slices from contrast field (CF) and working memory (WM) at the offset of the memory array (note the corresponding boxes in panel A). (E and F) Time-slices from CF and WM during the presentation of the test array (note the corresponding boxes in panel A). In this trial, a different color value is presented during the test array (note the above-threshold activation in panel E) and the model responds “different” (note the activation profile of the decision nodes in panel B). See the online article for the color version of this figure.

Figure 4.

Model performing different trial types. (A) The model correctly performing a “same” trial. At the offset of the memory array, the working memory (WM) field has built peaks corresponding to the four items in the memory array. During test when the same item are presented, activation in contrast field (CF) stays below threshold (note the asterisks above CF). Here, the model responds “same” (note the activation of the decision nodes). (B) The model correctly performing a “different” trial. Now, during the test array, a new item is presented that goes above threshold in CF (note the asterisk above CF). (C) The model performing a same trial but generating an incorrect response. At the offset of the memory array, the WM field has failed to consolidate one of the items into memory (note the asterisk above WM). Subsequently, during the presentation of the same items during the test array, the corresponding stimulus goes above threshold in CF (note the asterisk above CF) and the model generates a different response. (D) The model performing a different trial but generating an incorrect response. In this example, the model has overly robust activation with the WM field which leads to stronger inhibition within CF and a failure of the new item to go above threshold in CF (note the asterisk above CF). See the online article for the color version of this figure.

Figure 5.

Illustration model activation dynamics and hemodynamics. (A, B, D, and F) Stimulated local field potential (solid lines) and corresponding hemodynamic responses (dashed lines) from the “same” node (A), “different” node (B), contrast field (CF; D) and working memory (WM; F). (C, E, and G) Activation of model components over a series of eight trials (note the labels at the bottom that categorize each trial type) for the decision nodes (C), CF (E), and WM (G). LFP = local-field potential; HDR = hemodynamic response. See the online article for the color version of this figure.

Figure 6.

Simulations of Todd and Marois (2004). (A) Behavioral performance and model simulations. (B) Blood oxygen level dependent (BOLD) response from intraparietal sulcus (IPS) across memory loads of 1, 2, 3, 4, 6, and 8 items (left) and simulated hemodynamic response from contrast field (CF) layer (right). (C) Simulated hemodynamic response from the other three components of the model. DF = Dynamic Field; WM = working memory. See the online article for the color version of this figure.

Figure 7.

Simulations of Magen et al. (2009). (A) Behavioral performance and model simulations. (B) Blood oxygen level dependent (BOLD) response from posterior parietal cortex (PPC) across memory loads of 1, 3, 5, and 7 items (left) and simulated hemodynamic response from contrast field (CF) layer (right). (C) Simulated hemodynamic response from the other three components of the model. DF = Dynamic Field; WM = working memory. See the online article for the color version of this figure.

Figure 8.

Task design and behavioral/simulation data. (A) A trial began with a sample array consisting of 2, 4, or 6 colored items. Next came a retention interval and presentation of a test array. On change trials (50% of trials), one randomly selected item was shifted 36° in color space. (B) Percent correct from behavioral study. (C) Percent correct from functional magnetic resonance imaging (fMRI) study. In both studies, there were many errors at set-Size 4, but performance was above chance, t(27) = 23.5, p < .001. (D) Simulations reproduced the behavioral pattern. Error bars show ±1 SD. See the online article for the color version of this figure.

Figure 9.

Average amplitude of hemodynamic response across model components and trial types. This figure shows the variations in the amplitude of the hemodynamic response when performing our version of the change detection task (correct change trial = hit; correct same trial = correct-rejection [CR]; incorrect change trial = Miss; incorrect same trial = false alarm [FA]). CF = contrast field; WM =working memory. See the online article for the color version of this figure.

Figure 10.

Mapping of model components to regions of interest (ROIs). (A) Yellow spheres show ROIs that corresponded to contrast field (CF) and red spheres show ROIs that corresponded to WM + “same” (WM, working memory). (B–C) Time-course plots showing the blood oxygen level dependent (BOLD) response and predicted time-courses from the Dynamic Field (DF) model and from the accuracy categorical model within regions that were mapped by DF components. A participant is shown that preferred the DF model (P1) and a participant that preferred the accuracy categorical model (P8). (D) The same time-courses and participants are shown within a region that was not mapped by a DF component. (E) Scatter plot showing the correlation between participant-specific weights of the working memory (WM) component from the DF model to rTPJ (temporoparietal junction) activation and individual capacity. See the online article for the color version of this figure.

Figure 11.

Relative model precision. Average improvement in model precision for the Dynamic Field (DF) model relative to the array of categorical models. Top panel shows relative improvement in model precision within the 23 ROIs (regions of interest). Yellow (contrast field, CF) and red (working memory, WM + “same”) arrows mark regions that were mapped to components of the DF model. Bottom panel shows relative improvement in model precision by participant. Arrows indicate participants that preferred a categorical model over the Dynamic Field (DF) model with four components. Gray arrows indicate participants that switched to prefer the DF model when only three components of the DF model were included. See the online article for the color version of this figure.

Figure 12.

Activation and model prediction across trial-types within lIPS. Activation (solid) and model predictions for the Dynamic Field (DF; dotted) and change categorical (dashed) models is plotted across trial-types and different set sizes. Left graphs represent activation for a participant that preferred the DF model. Right graphs represent activation for a participant that preferred the change categorical model. The bar graphs show the average absolute difference between activation and model predictions within the 10 s time window. CR = correct-rejection. See the online article for the color version of this figure.

Figure 13.

Relative differences between activation in lIPS and model predictions by trial-type. We first calculated the absolute average difference between activation and model predictions for the Dynamic Field (DF) model, accuracy categorical model, and change categorical model within a 10 s window for each trial type (as visualized in Figure 12). Next, the difference for the DF model was subtracted from the difference of each categorical model. Positive values, then, reflect instances where the categorical model deviated from observed activation more so than the DF model. Negative values indicate instances in which the DF model deviated from observed activation more so than the categorical model. CR = correct-rejection; FA = false alarm.