Visual Information Routes in the Posterior Dorsal and Ventral Face Network Studied with Intracranial Neurophysiology and White Matter Tract Endpoints

Abstract

Occipitotemporal regions within the face network process perceptual and socioemotional information, but the dynamics and information flow between different nodes of this network are still debated. Here, we analyzed intracerebral EEG from 11 epileptic patients viewing a stimulus sequence beginning with a neutral face with direct gaze. The gaze could avert or remain direct, while the emotion changed to fearful or happy. N200 field potential peak latencies indicated that face processing begins in inferior occipital cortex and proceeds anteroventrally to fusiform and inferior temporal cortices, in parallel. The superior temporal sulcus responded preferentially to gaze changes with augmented field potential amplitudes for averted versus direct gaze, and large effect sizes relative to other network regions. An overlap analysis of posterior white matter tractography endpoints (from 1066 healthy brains) relative to active intracerebral electrodes in the 11 patients showed likely involvement of both dorsal and ventral posterior white matter pathways. Overall, our data provide new insight into the timing of face and social cue processing in the occipitotemporal brain and anchor the superior temporal cortex in dynamic gaze processing.

Keywords: emotion; face; gaze; iEEG; occipitotemporal cortex.

Figure 1

Experimental paradigm. A trial began with a fixation cross that was replaced by a neutral face with direct gaze (Face 1). After a variable delay, the face turned into happy or fearful with or without gaze aversion (Face 2), in an apparent motion manipulation. In 89% of the trials, a checkerboard would then appear on the left or right of the face. The patient had to press a button as quickly as possible after the appearance of the target checkerboard.

Figure 2

Occipitotemporal cortex sampling across the 11 patients. (A) Bipolar sites across patients. The sagittal, axial, and coronal views of the brain illustrate the coverage obtained with the 323 bipolar sites retained for analysis across the 11 patients. The sites are color-coded as a function of the patient to which they belong. (B) Bipolar sites across ROIs. The same sites as in A are depicted, except that here they are color-coded as a function of the ROI in which they were localized: IOC, Inferior Occipital Cortex; FC, Fusiform Cortex; ITC, Inferior Temporal Cortex; STC, Superior Temporal Cortex. The number (n) of bipolar sites comprised in each ROI is indicated in parentheses. (C) Single patient illustration of bipolar site localization. These (normalized) MRI images show some of the bipolar sites responding to faces for Patient 1 and Patient 10. For visualization purposes, these have been projected to the same x- or y-coordinate. The color and shape code the corresponding ROI.

Figure 3

Responsiveness to face onset (Face 1) and change (Face 2) in the four regions of interest. (A) Number of responsive bipolar sites in each ROI. The locations of the responsive sites to Face 1, Face 2, or both are represented on 3D axial brain view at the left of each ROI bar plot. For each ROI, the bar plots represent the total number of unresponsive sites (in gray), the number of sites responding only to Face 1 (in dark blue), only to Face 2 (in red), and to both Face 1 and Face 2 (in lavender). The responsiveness profile varied across ROIs. The IOC was the most responsive region and the STC showed a relative preference for Face 2 (See Supplementary Results S2.3 for details). (B) Distribution of unresponsive and responsive sites along the posterior–anterior axis. The left plot indicates the posterior–anterior span across the y-coordinate of each of the four ROIs. The remaining four plots represent the location (y-coordinate) of the unresponsive and responsive sites in each ROI, pooling together the right and left hemisphere sites. Each color-coded dot represents a site, color-coded as in A. IOC, Inferior Occipital Cortex; FC, Fusiform Cortex; ITC, Inferior Temporal Cortex; STC, Superior Temporal Cortex.

Figure 4

ERPs to Face 1 and Face 2 across the four ROIs in the right hemisphere. (A) ERP morphology. ERPs in response to Face 1 (in blue) and Face 2 (in red) were averaged across patients, within each ROI, for each slice (A–F) along the posterior-to-anterior axis (for FC, ITC, and STC). Bipolar sites for IOC were clustered in a narrow y-range and were therefore averaged altogether. The number of sites (n) averaged for each ERP is indicated on the bottom right-hand corner of each plot. The gray area around ERP time courses represents the standard error of the mean (no gray area for ERPs obtained from a single site). The amplitude of ERPs is expressed in z-scores and the scale is the same for all plots. See Supplementary Figures 2–5, for a complete illustration of all sites included in each slice. (B) ERP amplitude. The absolute effect size (Cohen’s d) of the N200 to Face 1 (blue) and Face 2 (red) is shown for IOC (first row), FC (second row), and ITC (third row); for STC (fourth row), effect size was computed over the maximum of the slow ERP deflection to Face 1, while the peak of the N200 was considered for Face 2. We computed effect sizes over the slices showing the clearest ERPs (IOC: all sites; FC: slices A–E; ITC: slices B–D; STC: slices C–E). For each ROI, we statistically tested the difference in effect sizes between Face 1 and Face 2. The error bars represent the standard error of the mean. The dotted lines represent commonly accepted evaluations of the effect size measure. (C) Peak latency. The latency of the N200 to Face 1 (blue) or Face 2 (red) is represented for all ROIs. As in B, it was computed from the slices showing the clearest ERPs. The latency computed for ERPs to Face 1 in the STC corresponds to the latency of the maximum of the slow ERP deflection. For each ROI, we statistically tested the difference in peak latency between Face 1 and Face 2, using a jackknife procedure. Error bars represent the standard error of the mean derived from this procedure. (D) ERP comparison across ROIs. A color-coded schematic representation of the location of the four ROIs is presented on the 3D sagittal view (left). The areas appearing in more intense color correspond to the slices where ERPs were analyzed (see B). The middle two panels depict ERP waveforms to Face 1 (left) and to Face 2 (right) that were obtained by averaging ERPs in each ROI (taking into account the slices with the clearest ERPs as in B and C). In the rightmost display panel, two tables (for Face 1 and Face 2, respectively) show the results of the statistical comparisons of peak latencies across each pair of ROIs. IOC, Inferior Occipital Cortex; FC, Fusiform Cortex; ITC, Inferior Temporal Cortex; STC, Superior Temporal Cortex. NS: nonsignificant; (^*): P < 0.08; ^*: P < 0.05; ^**: P < 0.01; ^***: P < 0.005.

Figure 5

Effect Sizes for Emotion and Gaze for each of the four ROIs. For each ROI (IOC, FC, ITC, and STC), each dot corresponds to one bipolar site, significantly responding to Emotion (on the left) or Gaze (on the right), with the corresponding effect size (absolute Cohen’s d) represented on the y-axis. The dark gray open circle represents the mean effect size across sites, for Emotion and Gaze, respectively, within each ROI. Effect sizes were compared between Emotion and Gaze in each ROI (gray bars between the open circles) and across ROIs for each Emotion and Gaze effect (top gray bars). The dotted lines represent commonly acceptable evaluations of the effect size values. The responsive site location can be viewed on the 3D axial brain silhouette views, for each effect in each ROI (bottom row), and the corresponding patient number is identifiable with the dot color (see color–patient correspondence on the bottom right). IOC, Inferior Occipital Cortex; FC, Fusiform Cortex; ITC, Inferior Temporal Cortex; STC, Superior Temporal Cortex. NS: nonsignificant; ^*: P < 0.05; ^**: P < 0.01.

Figure 6

ERPs from sites in the Superior Temporal Cortex region (STC) responding to Gaze. The patient, electrode, and site number, as well as the anatomical localization and the MNI coordinates are indicated for each ERP. Horizontal green bars indicate the time window where the conditions significantly differ. The coronal MRI figures show the localization of each site, on the patient’s normalized brain, at the corresponding y-coordinate. Black diamonds near x-axis indicate polarity inversions in adjacent sites. MTG: Midtemporal Gyrus, STS: Superior Temporal Sulcus. ^*: P < 0.05, ^***: P < 0.005, Monte Carlo P values.

Figure 7

Effects of Gaze and Emotion within the four ROIs in Patient 17. This patient was selected due to extensive occipitotemporal sampling and because he displayed significant effects across all four ROIs; we selected representative electrode shafts in each ROI for this patient in order to illustrate both the effects of Gaze and the effects of Emotion (see Supplementary Figs 8–11 for all effects, over all patients). The STC showed the most marked differences between averted and direct gaze conditions (top line of plots), with larger ERPs for averted gaze. The fearful versus happy emotion conditions produced subtle but quite long-lasting ERP changes that could occur across the entire ERP time course, at least in the FC and IOC of this patient. The sites are projected on coronal views of the patient’s postimplantation structural MRI. Note that sites 1 and 2 of Elec 9 in FC are not visualized because they were located in coronal slices different from sites 3 to 5. Black (solid and open) diamonds on top of the y-axis of the plots indicate polarity inversions between adjacent sites. ERP polarities were not rectified in these plots, in order to visualize polarity inversions. FC, Fusiform Cortex; IOC, Inferior Occipital Cortex; ITC, Inferior Temporal Cortex; STC, Superior Temporal Cortex. ^*: P < 0.05, ^**: P < 0.01, ^***: P < 0.005, corrected-over-time Monte Carlo P values.

Figure 8

Endpoints for canonical white matter tracts that course partly, or wholly, through occipitotemporal regions. A series of inflated cortical surfaces display gray matter endpoints for the middle longitudinal fasciculus (MdLF) of the superior parietal lobule (SPL) and the angular gyrus (Ang), arcuate fasciculus (Arc), vertical occipital fasciculus (VOF, D = dorsal, V = ventral), posterior Arc (pArc), inferior longitudinal fasciculus (ILF), superior longitudinal fasciculus (SLF) subcomponents 1 and 2, and the temporoparietal connection of the superior parietal lobule (TP-SPL). Color scales display the number of subjects showing voxels at these endpoints in a density histogram that ranges from 1 to 1000 (red through to pink). Data have been thresholded at 150.

Figure 9

Overlap analysis between posterior white matter tract endpoints and active bipolar iEEG sites. The rows in the matrix are the ROIs (IOC, ITC, FC, STC, and IPS), with one row for the sites that responded to Face 1 (including the sites that responded to Face 1 only and the sites that responded to both Face 1 and Face 2, see Materials and Methods; labeled as Face 1) and one row for the sites that responded to Face 2 only (labeled as Face 2), for every ROI except IPS where no site was found to respond to Face 2 only. The columns are the white matter tract endpoints. The color scale indicates the proportion of the active sites that lay in a given tract endpoint zone. Inf, Sup, Post, Ant indicate the inferior and superior parts or the posterior and anterior parts (respectively) of the indicated tracts; arcuate fasciculus (Arc), inferior longitudinal fasciculus (ILF), angular gyrus and superior parietal lobule sections of the middle longitudinal fasciculus (MdLF-Ang and MdLF-SPL, respectively), superior longitudinal fasciculus (SLF) (where 1, 2, and 3 denote the 3 subcomponents of the SLF; see Thiebaut de Schotten et al. 2011), temporoparietal connection of the superior parietal lobule (TP-SPL), vertical occipital fasciculus (VOF), and posterior arcuate (pArc).

Figure 10

Putative routes of information flow for faces in the brain. (A) Structures of the core (pink) and extended (blue) face network based on Gobbini and Haxby (2007) and Haxby et al. (2000). ant, anterior; post, posterior; inf, inferior; OFA, occipital face area; FFA, fusiform face area; STS, superior temporal sulcus; TPJ, temporoparietal junction. (B) Putative hierarchy of information flow within the face network showing known direct white matter connections (solid lines) (adapted from Grill-Spector et al. 2017). The nature of connections between other brain regions, including the posterior STS (part of the core face network) and amygdala (extended face network) are not known. Legend: similar to part A, IPS, intraparietal sulcus; V1-V2, early visual areas. (C) White matter pathways that may be involved in routing information within the posterior visual face pathway, based on overlap analysis of white matter tract endpoints in 1066 healthy subjects and coordinates of active bipolar sites in patients with epilepsy. (Modified from Bullock et al. 2019). SLF, superior longitudinal fasciculus; TP-SPL, temporoparietal connection of the superior parietal lobule; Arc, arcuate fasciculus; pArc, posterior arcuate fasciculus; ILF, inferior longitudinal fasciculus; VOF, vertical occipital fasciculus; MdLF-Ang, middle longitudinal fasciculus branch of the angular gyrus; MdLF-SPL middle longitudinal fasciculus branch of the superior parietal lobule. We also included Meyer’s loop (Meyer’s) that is the optic radiation connecting the lateral geniculate nucleus and the occipital lobe in this schematic figure. (D) Putative routes of information flow evidenced by the multimodal data integration of neurophysiological data and white matter tract endpoints overlap analysis. The solid lines represent the putative routes for which an overlap between both ends of tract endpoints and active sites was observed, while broken lines indicate connections with overlap at one end of the tract only, in this study. Note that short-range fibers that may be key in information flow across the ventral occipitotemporal ROIs (from IOC to ITC and to FC) and could also play a role in connecting FC, ITC, and STC were not included in our tract endpoint analysis and are not represented here.