Using stochastic language models (SLM) to map lexical, syntactic, and phonological information processing in the brain

Abstract

Language comprehension involves the simultaneous processing of information at the phonological, syntactic, and lexical level. We track these three distinct streams of information in the brain by using stochastic measures derived from computational language models to detect neural correlates of phoneme, part-of-speech, and word processing in an fMRI experiment. Probabilistic language models have proven to be useful tools for studying how language is processed as a sequence of symbols unfolding in time. Conditional probabilities between sequences of words are at the basis of probabilistic measures such as surprisal and perplexity which have been successfully used as predictors of several behavioural and neural correlates of sentence processing. Here we computed perplexity from sequences of words and their parts of speech, and their phonemic transcriptions. Brain activity time-locked to each word is regressed on the three model-derived measures. We observe that the brain keeps track of the statistical structure of lexical, syntactic and phonological information in distinct areas.

Fig 1. Lexical stream.

Left, right and ventral view of inflated cortex plot of the lexical stream.

Fig 2. Syntactic stream.

Left, right and ventral view of inflated cortex plot of the syntactic stream.

Fig 3. Phonological stream.

Left, right and ventral view of inflated cortex plot of the phonological stream.

Fig 4. Streams comparison and overlap.

Inflated cortex view of the maps of the lexical (green), syntactic (blue) and phonological (red) streams. In this view the overlap between lexical and syntactic streams is particularly evident in the right Middle Temporal Lobe and in the left posterior Superior Temporal Gyrus (cyan). Overlap between syntactic and phonological streams is also evident in the bilateral Angular Gyrus (violet).