Individual differences in the neural dynamics of visual narrative comprehension: The effects of proficiency and age of acquisition

Abstract

Understanding visual narrative sequences, as found in comics, is known to recruit similar cognitive mechanisms to verbal language. As measured by event-related potentials (ERPs), these manifest as initial negativities (N400, LAN) and subsequent positivities (P600). While these components are thought to index discrete processing stages, they differentially arise across participants for any given stimulus. In language contexts, proficiency modulates brain responses, with smaller N400 effects and larger P600 effects appearing with increasing proficiency. In visual narratives, recent work has also emphasized the role of proficiency in neural response patterns. We thus explored whether individual differences in proficiency modulate neural responses to visual narrative sequencing in similar ways as in language. We combined ERP data from 12 studies examining semantic and/or grammatical processing of visual narrative sequences. Using linear mixed effects modeling, we demonstrate differential effects of visual language proficiency and "age of acquisition" on N400 and P600 responses. Our results align with those reported in language contexts, providing further evidence for the similarity of linguistic and visual narrative processing, and emphasize the role of both proficiency and age of acquisition in visual narrative comprehension.

Keywords: Age of acquisition; Comics; Event-related potentials; Proficiency; Visual language; Visual narratives.

Fig. 1

Scatterplots of N400 effect magnitudes against P600 effect magnitudes, for (a) all contrasts and (b) each contrast type. The dashed line represents equal N400 and P600 magnitudes. Values above the dashed line indicate a negativity-dominance, i.e., participants primarily show an N400 effect to the various contrast types. Values below the dashed line indicate a positivity-dominance, i.e., participants primarily show a P600 effect to the various contrast types. The solid diagonal lines indicate the best-fit line from the correlation analysis for each contrast type. Each dot represents one participant

Fig. 2

Interaction plots for the models of N400 effects (top row) and P600 effects (bottom rows), broken up by high/low VLFI (left panels), and early/late ASCR (right panels)

Fig. 3

Scatterplot of N400 effect against ASCR for each of the three contrasts. Each dot represents one participant

Fig. 4

Scatterplot of P600 effect against ASCR for each of the three contrasts. Each dot represents one participant