Stochastic transitions between neural states in taste processing and decision-making

Abstract

Noise, which is ubiquitous in the nervous system, causes trial-to-trial variability in the neural responses to stimuli. This neural variability is in turn a likely source of behavioral variability. Using Hidden Markov modeling, a method of analysis that can make use of such trial-to-trial response variability, we have uncovered sequences of discrete states of neural activity in gustatory cortex during taste processing. Here, we advance our understanding of these patterns in two ways. First, we reproduce the experimental findings in a formal model, describing a network that evinces sharp transitions between discrete states that are deterministically stable given sufficient noise in the network; as in the empirical data, the transitions occur at variable times across trials, but the stimulus-specific sequence is itself reliable. Second, we demonstrate that such noise-induced transitions between discrete states can be computationally advantageous in a reduced, decision-making network. The reduced network produces binary outputs, which represent classification of ingested substances as palatable or nonpalatable, and the corresponding behavioral responses of "spit" or "swallow". We evaluate the performance of the network by measuring how reliably its outputs follow small biases in the strengths of its inputs. We compare two modes of operation: deterministic integration ("ramping") versus stochastic decision-making ("jumping"), the latter of which relies on state-to-state transitions. We find that the stochastic mode of operation can be optimal under typical levels of internal noise and that, within this mode, addition of random noise to each input can improve optimal performance when decisions must be made in limited time.

Figure 1.

Architecture of model network for taste processing. A, The model network is based on five pools of neurons, with each pool containing N_E excitatory cells and N_I = N_E/4 inhibitory cells (by default, N_E = 80, N_I = 20). Recurrent excitatory connections (data not shown) dominate within a pool. Arrows indicate significant excitatory connections between pools, and balls indicate the strong cross-inhibition between pools that produce one binary decision of palatability. Other connections (data not shown) produce weak inhibition between all pools. Recurrent excitation is sufficient to maintain the activity of a pool, once a combination of inputs and random fluctuations has driven the pool into an active state. The network progresses through the three epochs of detection (somatosensory response), identification (chemosensory response), and decision (palatability response) that are observed in gustatory cortex during taste processing. Each successive epoch is driven by strong activity arising in a particular pool, which through cross-connections alters the activity in all pools. B, C, Trial-averaged firing rates of four cells in the model network, after a stimulus representing either taste 1 (B) or taste 2 (C). Color of the trace indicates the pool from which the cell is taken. Note the gradual ramping of firing rates that can take >1 s for some cells.

Figure 2.

Neural activity produces sharp transitions between discrete states, with trial-to-trial variability in transition times. A, Spike trains of 10 cells (2 from each pool) with the results of a hidden Markov analysis of their activity after a stimulus representing taste 1. Bold solid lines over the solid color indicate the probability of the network being in a given state. The probability remains close to 1, except for short intervals when the system transitions to a new state. B, A subsequent trial with the same inputs representing the same taste gives rise to an identical sequence of states but with different transition times. C, Distribution of firing rates for each of the 10 cells in the four successive HMM states produced by taste 1. D, E, As in A but for the response of the same cells to a stimulus representing taste 2. F, As in C but for the firing rates in the five successive HMM states produced by taste 2.

Figure 3.

Transition-triggered average reveals more rapid changes in firing rate than apparent in standard histograms. A, In orange, we plot a histogram of firing rates of a cell, aligned to the end of the second HMM state of each trial. This transition-triggered average (TTA) of firing rate reveals a rapid change in rate at the time of transition that are smoothed out in the standard histogram (yellow line, which is cell 2, taste 1 from Fig. 1B,C shifted for comparison). B, Cell 8, taste 1. Blue indicates TTA. C, Cell 2, taste 2. Orange indicates TTA. D, Cell 10, taste 2; maroon indicates TTA.

Figure 4.

Detailed architecture of the decision-making part of the network, used alone for additional analysis, with its two modes of operation. A, All of the types of connection within and between pools are represented in the figure, with arrows indicating excitatory connections and balls indicating inhibitory connections. Size of each pool ranges from 20 excitatory and 5 inhibitory cells to 80 excitatory and 20 inhibitory cells. As shown, the inputs to the network produce a bias toward swallow (3 arrows vs 2, indicates a stronger input; in practice, the stronger input is just 10% higher, unless otherwise stated). B, C, Pseudopotentials indicating the jumping mode of decision-making, which arises with reduced total input, or by increasing the inhibitory conductance, without a bias (B) and with a bias toward swallow (C). Initial state is deterministically stable, so fluctuations are required to produce a decision. D, E, Pseudopotentials indicating the ramping mode of decision-making with sufficient external input or without inhibitory current, without a bias (D) and with a bias toward swallow (E).

Figure 5.

Hidden Markov analysis of decision-making network in two modes of operation. All figures include the spike trains of eight selected cells (4 from each decision-making pool) on a particular trial, combined with the HMM output of probability for each state as a function of time (solid curves with color below curve). A, B, Two separate trials for a network in ramping mode. Note the relatively slow transitions between states in the HMM output. C, D, Two separate trials from a network in jumping mode. Transitions are much sharper. Networks were matched for similar average temporal behavior but with different parameters. Ramping mode (A, B): g_L = 18 nS, W_EE = 5, W_EI = 10, W_IE = 6, α = 0.25 (performance of 37 with N = 50). Jumping mode (C, D): g_L = 22 nS, W_EE = 5, W_EI = 30, W_IE = 8, W_II = 3, α = 0.5 (performance of 41 with N = 50). These different parameters account for differences in behavior: stimuli are identical.

Figure 6.

Decision-making network produces a winner-takes-all response to a small bias in input. A, B, Network in ramping integration mode (parameters of Fig. 5A,B). A, In a single trial, solid curves indicate activity of each excitatory population, and dashed curves indicate activity the inhibitory population of the corresponding color. B, Activity of the two excitatory populations across 10 trials, given a small bias current to the “choose swallow” population (green) over the “choose spit” population (red). An error trial is visible when the red curve reaches high activity. C, D, Network operating in jumping mode (parameters of Fig. 5C,D). C, A single trial with average activity of excitatory populations solid and their corresponding inhibitory population in the same color dashed. D, Ten trials of activity of the two excitatory populations given a small bias to the swallow population (green). An error trial is visible when the choose spit population (red) achieves high activity. On one trial, neither population reaches high activity, representing an undecided state.

Figure 7.

Benefits and limits of noise and the stochastic, jumping mode of decision-making in a fixed time interval. A, Total applied current is varied to shift the network from the ramping mode (high total applied current) to the jumping mode of decision-making (low total applied current) while maintaining a fixed bias (difference between the currents to the two populations is held fixed). Proportion of trials in which the winning pool is the one with greater input, maximized when the system is in jumping mode. B, With constant inputs for a limited, 2 s decision-making period, inhibitory conductance is increased to shift the network to jumping mode. Optimal performance occurs in the parameter region of the jumping mode. Bottom curve, Inputs are fixed currents. Top curve, Inputs are Poisson spike trains with the same mean current as the bottom curve. Note that performance is better when inputs are noisy. C, Performance varies with size of network, always improving with network size (which reduces internal noise) for deterministic integration in the ramping mode but having an optimal level for the jumping mode. Note that noise is uncorrelated across cells in these simulations, so unrealistically low levels of internal variability could be reached with large network size.

Figure 8.

Optimal mode for decision-making shifts to ramping with strong external signal, low internal noise, and short duration of stimuli. A–C, Response of the network with a doubling of both inputs used in Figure 7, B and C. Network size increases from left to right panels: A, n = 50; B, n = 100; C, n = 200. D–F, Response of the network with a factor of 5 increase in both inputs of Figure 7, B and C. Network size increases from left to right panels: A, n = 50; B, n = 100; C, n = 200. In A–F, input durations are 1 s (red), 2 s (green), or 5 s (blue). The transition between jumping and ramping modes is determined by a switch from increasing to decreasing variance of response times with reduced internal noise (larger networks).

Figure 9.

Nullcline analysis reveals how the jumping mode reduces response variability arising from non-identical initial conditions. Nullclines in the synaptic variables indicate the steady states of the system. Green lines indicate dS1/dt = 0 for a given S2; red lines indicate dS2/dt = 0 for a given S1. Filled circles are stable fixed points of the system, and open circles are unstable fixed points. Trajectories for the system with inputs biased by 10% in favor of S2 from a symmetric range of starting points are presented without noise (A–C) and with noise (D–F). Orange trajectories result in a correct response with high S2, low S1; magenta trajectories show an incorrect response with low S2, high S1; blue trajectories result in a lack of response with low S1 and S2. A–D, With inputs of 0.0040 and 0.0044, the low activity state is stable; B–E, with inputs of 0.0050 and 0.0055, the low activity state is just unstable and noise generates many errors; C–F, with inputs of 0.0200 and 0.0220, the low activity state is highly unstable and asymmetry in the initial state of the system produces multiple errors.