Computational framework for investigating predictive processing in auditory perception

Abstract

Background: The brain tracks sound sources as they evolve in time, collecting contextual information to predict future sensory inputs. Previous work in predictive coding typically focuses on the perception of predictable stimuli, leaving the implementation of these same neural processes in more complex, real-world environments containing randomness and uncertainty up for debate.

New method: To facilitate investigation into the perception of less tightly-controlled listening scenarios, we present a computational model as a tool to ask targeted questions about the underlying predictive processes that connect complex sensory inputs to listener behavior and neural responses. In the modeling framework, observed sound features (e.g. pitch) are tracked sequentially using Bayesian inference. Sufficient statistics are inferred from past observations at multiple time scales and used to make predictions about future observation while tracking the statistical structure of the sensory input.

Results: Facets of the model are discussed in terms of their application to perceptual research, and examples taken from real-world audio demonstrate the model's flexibility to capture a variety of statistical structures along various perceptual dimensions.

Comparison with existing methods: Previous models are often targeted toward interpreting a particular experimental paradigm (e.g., oddball paradigm), perceptual dimension (e.g., pitch processing), or task (e.g., speech segregation), thus limiting their ability to generalize to other domains. The presented model is designed as a flexible and practical tool for broad application.

Conclusion: The model is presented as a general framework for generating new hypotheses and guiding investigation into the neural processes underlying predictive coding of complex scenes.

Keywords: Bayesian inference; Predictive coding; Statistical inference; neural decoding; uncertainty.

Figure 1:

Model description. a) The model uses multiple context hypotheses to account for unknown changes in the observed sequence. Context-specific predictions ${\overrightarrow{P}}_t$ based on sufficient statistics ${\overrightarrow{\mathrm{\Theta }}}_t$ are combined, weighted by corresponding beliefs ${\overrightarrow{B}}_t$, to yield the predictive distribution Ψ_t for the next input x_t+1. b) Upon observing x_t+1, the predictions and new input are used to update the statistics and beliefs, which are used in turn to predict the next input, and so on. There are three principal outputs from the model at each time: the surprisal of the newly observed input based on its prediction, the predictive distribution for the next input, and the beliefs (or posterior distribution over contexts). c) Outputs from the model for the example sequence in a). The top panel shows the predictive distribution at each time (in blue) with the input sequence overlaid, the middle panel shows the context beliefs, with each row corresponding to a particular context hypothesis c_i, and the bottom panel shows the surprisal for each input observation. Note the predictive distribution and context beliefs reflect the underlying change in statistics inferred by the model.

Figure 2:

Model outputs for example inputs from real-world audio clips. Each panel displays the model predictive distribution (top), context beliefs (middle), and surprisal (bottom) over time, with the input sequence overlaid on the predictive distribution (top, in black). The input dimension (feature), distributional choice in the model, and audio event annotation are indicated above. Audio clips can be found in Supplemental Information.