The brain's router: a cortical network model of serial processing in the primate brain

Abstract

The human brain efficiently solves certain operations such as object recognition and categorization through a massively parallel network of dedicated processors. However, human cognition also relies on the ability to perform an arbitrarily large set of tasks by flexibly recombining different processors into a novel chain. This flexibility comes at the cost of a severe slowing down and a seriality of operations (100-500 ms per step). A limit on parallel processing is demonstrated in experimental setups such as the psychological refractory period (PRP) and the attentional blink (AB) in which the processing of an element either significantly delays (PRP) or impedes conscious access (AB) of a second, rapidly presented element. Here we present a spiking-neuron implementation of a cognitive architecture where a large number of local parallel processors assemble together to produce goal-driven behavior. The precise mapping of incoming sensory stimuli onto motor representations relies on a "router" network capable of flexibly interconnecting processors and rapidly changing its configuration from one task to another. Simulations show that, when presented with dual-task stimuli, the network exhibits parallel processing at peripheral sensory levels, a memory buffer capable of keeping the result of sensory processing on hold, and a slow serial performance at the router stage, resulting in a performance bottleneck. The network captures the detailed dynamics of human behavior during dual-task-performance, including both mean RTs and RT distributions, and establishes concrete predictions on neuronal dynamics during dual-task experiments in humans and non-human primates.

Figure 1. Network architecture.

Schematic of the spiking neuron network model. Each population, represented with a circle, contains between 80 and 640 neurons. Circles with diagonal textures indicate inhibitory populations and all other circles indicate populations of pyramidal cells. Whenever two populations of neurons are connected this indicates full connectivity between them. The network includes two sensory modalities (sensory 1 and 2), organized in a hierarchy in which each successive layer receives inputs – mediated by rapid (time constant of 2ms) AMPA receptors - from various populations of the previous layer thus generating progressively more complex receptive fields. Each stimulus (for example, S1) is represented by the co-activation of four specific neural populations in the first layer of the sensory hierarchy. Just for illustration purposes, each stimulus is represented as a solid circle and the different features of this stimulus as parts of this circle, i.e. the 4 red neurons in the first layer represent a stimulus when they are active together. Sensory modules are also connected through non-specific feedback connections mediated by slow (time constant of 100ms) NMDA receptors. Both sensory modalities converge to the router, which is a common integrator. The integrator neurons feed back to the sensory neurons, generating recurrent activity which can maintain and amplify sensory information. Integrator neurons connect to response neurons and thus route information from sensory to motor neurons. Subsets of the neurons in the router link information from stimuli to responses in a flexible manner. Router neurons also receive input from task-setting neurons and thus act as detectors of the conjunction of the relevant task and the appropriate stimulus. The circuit involved in mapping S1 to R1 of Task 1 as well as the task-setting of Task 1 is emphasized in bold. Response execution is triggered by a set of bursting neurons that signal a threshold-cross of the input received from the routing neurons that integrate information. Response neurons feed back to the router and to inhibit the neurons immediately after the response. This inhibition prevents perseveration and is required to stabilize the network in a single response mode. In a typical PRP experiment, which we model here, subjects are instructed to respond to both tasks as fast as possible in a particular order. To enforce this response order in the network we organized the task-setting neurons in a hierarchy in which the neurons coding for Task 1 and Task 2 are controlled by a switch composed of task-order units (see Materials and Methods section for a detailed description).

Figure 2. Single-trial dynamics for task relevant and irrelevant stimuli.

Firing rates of representative trials of task-relevant (A–D) and task-irrelevant (E–H) stimuli. Each panel shows the firing rates averaged across a population (thick line) overlapped with spike rasters (each row of dots represent the spiking activity of a neuron in the population). Average firing rates were calculated by convolving the spike raster from a single trial with a gaussian filter of σ = 12ms. (A) Stimulus presentation (indicated with a dashed vertical line) generates a wave of activity that propagates through the successive stages of the sensory hierarchy. The colored circles represent the features coded by the various populations, following the notation of figure 1. (B) Router neurons show ramping activity until a response threshold is reached. (C) Activity in task-setting neurons is triggered by excitatory input from sensory neurons and is sustained for the duration of the task. (D) The response is signaled by a burst of excitatory neurons in the response network. (E–H) Same as panels A–D, but with the connections from sensory to task-setting areas removed. In the absence of projection from task-setting neurons (G) the activity in the router (F) does not reach the threshold to trigger a response in motor areas (H), despite strong activation in sensory areas (E).

Figure 3. Neural activations during dual-task performance.

(A–D) Firing rates in the dual-task condition inside the interference regime (SOA = 100ms). Each panel is defined as in Figure 2, with black and grey lines corresponding to populations from the first and second tasks, respectively. The specific populations plotted are indicated with colored circles, following the notation of Figure 1 (the circles to the left correspond to the first task). (E) Firing rates are plotted for 1200 trials (100 at each SOA) for neurons responding to the first and second tasks (left and right columns in each panel, respectively). White lines indicate the onset of the stimuli of task 1 and 2, and grey lines mark the specific times at which the average activity at each SOA crossed 1/3 of its peak value. In early sensory areas, both the onset and offset of the response are time-locked to stimulus presentation at short SOA, thus indicating a completely parallel mode of activation. In contrast, task-setting, response and integrating neurons show a highly serial activation profile. The onset and the offset of these neurons for task 2 in the interference regime are locked to the end of the corresponding process of task 1. In the non-interference regime, however, the onset of these neurons is locked to stimulus presentation. Higher sensory modules showed a hybrid profile, indicating that the same neuron can be involved in a phasic parallel response and also exhibits sustained activity until the response. In the interference regime, the onset of these neurons is locked to the presentation of the stimulus, but the offset show a sequential locking to the ending of task 1. Firing rates were calculated by filtering the instantaneous population firing rate with an exponential causal kernel with a time constant of 20 ms.

Figure 4. Mean response times: fingerprints of dual-task interference.

(A) Sketch of the PRP paradigm. Stimulus S1 is mapped to R1, and stimulus S2 to response R2. RT1 is defined as the time between S1 onset and the response R1. RT2 is defined as the time between the onset of S2 and the response R2. The SOA - defined as the time between onsets of S1 and S2 - is systematically varied, typically between 0 and 1000 ms. (B) Scheme of the mathematical formalism traditionally used to explain the delay in RT2 during the PRP. The vertical axis labels RT. The column on the left indicates the first task, and each colored box within the column represents a different stage of processing: Perceptual component (red), Central component (grey), and Motor component (blue). The series of columns on the right indicate the processing time for task 2 at different SOA, labeled on the x-axis. For each column, the three different boxes represent the three different stages of task 2: Perceptual component (green), Central component (grey), and Motor component (brown). As SOA progresses, the Perceptual component starts later. All components can be performed in parallel except for the Central component, which establishes a bottleneck. (C) Effect of SOA manipulations in response times for the proposed neural architecture. Average response times to the second task show a dependency on SOA similar to observations from PRP experiments: RT2 decreased with SOA within the interference range with a slope of −1, and is constant in the non-interference regime. RT1 is unaffected by SOA manipulations. In most PRP studies, response times are measured from the onset of the corresponding stimulus (T1 or T2). Other studies have used a different convention in which response times to both tasks are reported from trial onset (i.e., onset of T1). Here we show the PRP effect under both conventions, by defining the variable R2 = RT2 + SOA. The PRP effect is observed as an invariance of R2 with SOA for short SOA values, and a linear increase of R2 with SOA for large SOA values. Data points show averages across 300 trials. Error bars depict the standard error of the mean. (D–G) Effect of task complexity and SOA in response times. Each panel (containing two plots) defines the manipulation type (perceptual or central) and the affected task. Human data (taken from [18]) is shown to the left in each panel. To maintain the convention adopted in the experimental study , response times are shown relative to the onset of the first task. In each plot both easy (without manipulation, solid line) and difficult (with manipulation, dashed line) conditions are shown. RT1 is shown in grey, and R2 is shown in black. (D,F) We first varied the response complexity of the stimulus, changing the layer of the sensory hierarchy which feeds the integrator (Perceptual). This effect resulted in an increase of (RT1) when this factor affected the first task (F) which propagated to the second task (increase in RT2) within the interference regime. When the factor affected the second task (D) we observed no change in the first task, and a change in RT2 only outside of the interference regime, indicating that this manipulation can be absorbed during the PRP. This is exactly what is expected in the classic PRP model from a ‘pre-bottleneck’ manipulation . (E,G) We also varied the stimulus ambiguity (i.e. the relative input currents to each of the two competing sensory populations) (Central). When the ambiguity of the first task was increased (G), we observed an increase of (RT1) which propagated to the second task (increase in RT2) within the interference regime. When the factor affected the second task (E) we observed an effect on RT2 both inside and outside the interference regime. This is exactly what is expected in the classic PRP model from a ‘bottleneck’ manipulation . Data points show averages across 200 trials, except the baseline data (easy condition) that were averaged across 300 trials.

Figure 5. Response time distributions in dual-task execution.

(A and B) The model produces distributions of response times with a long tail. (B) As observed experimentally, for short SOA values (SOA = 0 ms in the figure), RT2 is more variable, since it concatenates the variances of both tasks. (C to E) Scatter plot of RT2 vs. RT1 for SOAs of 0ms (C), 150ms (D) and 500ms (E) (300 trials for each SOA). For short SOA values, the RTs are tightly correlated, a correlation that is caused by interference and sequentiality. (F) Cumulative RT distribution for varying SOA values. For increasing SOA values both the mean and the variance decrease. Simulated SOA values are indicated in the legend with arrows.

Figure 6. Response time sensitivity to stochastic fluctuations and low-frequency oscillations.

(A,B) Coefficients of the linear regression model used to relate fluctuations in background inputs to response time variability. Black traces correspond to stimulus-selective excitatory populations at different processing levels, as indicated in the figure's legend. Red traces correspond to inhibitory neurons within the same area. Shades depict 95% confidence intervals. A positive coefficient means that higher activity due to noise leads to faster responses. (A) Estimates for Task-1 sensory and router populations, with RT1 as the independent variable. The x-axis indicates the time relative to stimulus onset, and thus positive values correspond to noise fluctuations occurring after stimulus onset. (B) Estimates for Task-2 sensory and router populations, with RT2 as the independent variable. Here, neural activity across different trials was locked to response 1 before the regression analysis. (C) Mean response times (green trace) for single-task simulations as a function of the phase between stimulus onset and background noise. The x-axis depicts the phase of stimulus onset relative to the background fluctuation (brown trace, bottom), and the y-axis depicts the mean response time in milliseconds. Error bars indicate the standard error of the mean. 50 trials were simulated for each individual phase. Also shown in grey-scale are the response time histograms (bin size of 40 ms).

Figure 7. From the PRP to the attentional blink: masking effects on visibility.

When T2 is masked the model displays characteristic aspects of AB experiments. (A) Probability of responding correctly to T2 given T1 correct, for varying SOA (error bars depict the standard error of the mean). (B–D) Single-trial population firing rates of relevant populations for trials with seen and unseen T2. The average response of neurons selective to T2 (grey traces) and to the mask (purple traces) is shown at different sensory levels. The average activity of task-setting excitatory neurons selective to T1 is plotted in black, and motor neurons for the correct response to T2 are plotted in brown. The red line indicates the onset of T1, and the green line the onset of T2. (B) An unseen trial (the network fails to respond to T2) with SOA = 100 ms. The mask interferes with the reverberation process of T2 and activity in the last sensory areas decays before it can be recovered by the activation of T2 task-setting network. (C) A seen trial (the network responds) with the same SOA as in panel A (100 ms). The task-setting network disengages faster from T1, accelerating T2 task setting activation which permits integration before T2 sensory memory has faded out. (D) A trial with SOA = 100 ms and no mask (a PRP trial). In the absence of a backwards mask, the traces of T2 in sensory areas last longer and thus routing of T2 occurs despite the delayed engagement of task-setting neurons. Population firing rates were calculated by convolving the spike raster with an exponential causal kernel of 20 ms. (E) Average neural activations of selected populations for 129 seen and 64 not seen trials, for SOA = 100 ms. Each column indicates the averaged activity of one selective excitatory population, as indicated in the top of the panel, for the first (top row, red) and second (bottom row, green) tasks. The two traces shown in each cell show the activity of the same population when a correct (light green and red) or absent (dark green and red) response is given to T2. Averages were calculated by filtering the instantaneous population firing rate with an exponential causal kernel with a time constant of 20 ms and averaging across trials.