Combined eye tracking and fMRI reveals neural basis of linguistic predictions during sentence comprehension

Abstract

It is widely agreed upon that linguistic predictions are an integral part of language comprehension. Yet, experimental proof of their existence remains challenging. Here, we introduce a new predictive eye gaze reading task combining eye tracking and functional magnetic resonance imaging (fMRI) that allows us to infer the existence and timing of linguistic predictions via anticipatory eye-movements. Participants read different types of word sequences (i.e., regular sentences, meaningless jabberwocky sentences, non-word lists) up to the pre-final word. The final target word was displayed with a temporal delay and its screen position was dependent on the syntactic word category (nouns vs verbs). During the delay, anticipatory eye-movements into the correct target word area were indicative of linguistic predictions. For fMRI analysis, the predictive sentence conditions were contrasted to the non-word condition, with the anticipatory eye-movements specifying differences in timing across conditions. A conjunction analysis of both sentence conditions revealed the neural substrate of word category prediction, namely a distributed network of cortical and subcortical brain regions including language systems, basal ganglia, thalamus, and hippocampus. Direct contrasts between the regular sentence condition and the jabberwocky condition indicate that prediction of word category in meaningless jabberwocky sentences relies on classical left-hemispheric language systems involving Brodman's area 44/45 in the left inferior frontal gyrus, left superior temporal areas, and the dorsal caudate nucleus. Regular sentences, in contrast, allowed for the prediction of specific words. Word-specific predictions were specifically associated with more widely distributed temporal and parietal cortical systems, most prominently in the right hemisphere. Our results support the presence of linguistic predictions during sentence processing and demonstrate the validity of the predictive eye gaze paradigm for measuring syntactic and semantic aspects of linguistic predictions, as well as for investigating their neural substrates.

Keywords: Eye tracking; Language; Prediction; Syntax; fMRI.