Decoding of single-trial EEG reveals unique states of functional brain connectivity that drive rapid speech categorization decisions

Abstract

Objective: Categorical perception (CP) is an inherent property of speech perception. The response time (RT) of listeners' perceptual speech identification is highly sensitive to individual differences. While the neural correlates of CP have been well studied in terms of the regional contributions of the brain to behavior, functional connectivity patterns that signify individual differences in listeners' speed (RT) for speech categorization is less clear. In this study, we introduce a novel approach to address these questions.

Approach: We applied several computational approaches to the EEG, including graph mining, machine learning (i.e., support vector machine), and stability selection to investigate the unique brain states (functional neural connectivity) that predict the speed of listeners' behavioral decisions.

Main results: We infer that (i) the listeners' perceptual speed is directly related to dynamic variations in their brain connectomics, (ii) global network assortativity and efficiency distinguished fast, medium, and slow RTs, (iii) the functional network underlying speeded decisions increases in negative assortativity (i.e., became disassortative) for slower RTs, (iv) slower categorical speech decisions cause excessive use of neural resources and more aberrant information flow within the CP circuitry, (v) slower responders tended to utilize functional brain networks excessively (or inappropriately) whereas fast responders (with lower global efficiency) utilized the same neural pathways but with more restricted organization.

Significance: Findings show that neural classifiers (SVM) coupled with stability selection correctly classify behavioral RTs from functional connectivity alone with over 92% accuracy (AUC = 0.9). Our results corroborate previous studies by supporting the engagement of similar temporal (STG), parietal, motor, and prefrontal regions in CP using an entirely data-driven approach.

Figure 1:

(A, B) Demographic gender and age distributions. (C) Acoustic spectrograms of the speech stimuli: The stimulus continuum was created by parametrically changing vowel first formant frequency over five equal steps from 430 to 730 Hz (►), resulting in a perceptual-phonetic continuum from /u/ to /a/. (D) Token wise response times for auditory classification. Listeners are slower to label sounds near the categorical boundary (i.e., Token 3). Females (F) have significantly slower response times than males (M).

Figure 2:

Clustering RT data using GMM and BIC criteria. Model selection concerns both the covariance type and number of components in the model. Brute-force based empirical analysis shows that n=4 components with unique covariance matrix is optimal. The ‘*’ marked position of (A) shows the optimal combination. (B): Probability of trials loading into each component. (C): Token wise RT broken down by component. Based on behavioral RTs, four clusters are evident that distinguish subgroups of listeners based on their speech identification speeds: Fast (Cluster 1): 120~476 ms, Medium (Cluster 2): 478~722 ms, Slow (Cluster 0): 724~1430 ms, and Outliers (Cluster 3): 1432~2500 ms.

Figure 3:

The t-SNE embedded higher dimensional functional connectivity data are represented by a 2-dimensional scatter and kernel density estimation (KDE) plot. The green lines with ‘.’, blue lines with ‘*’, and red lines with ‘+’ sign represents data points for slow, medium, and fast RT participants, respectively.

Figure 4:

Effect of selection threshold on model performance prediction. The three x-labels represent (top) the range of each bin of features score (range: 0~1), (middle) the number of features falling in each bin, and (bottom) the corresponding percentage.

Figure 5:

Schematic diagram of the processing pipeline. The 64-ch EEG data is first preprocessed, and then source localization is adapted to convert skull surface data to cortical surface time series data (68 ROIs defined by the Desikan-Killany Atlas parcellation). Pairwise correlations were calculated to derive the connectivity matrix for each trial of the speech CP task. Behavioral response times (RTs) were clustered with Bayesian non-parametric (GMM) clustering. These clusters were labeled as Fast, Medium, and Slow RT. ANOVA analysis of Graph measures w adopted to test significance among RT groups. Stability selection and machine learning approaches were then used to find significant properties of the brain’s functional connectivity related to behavioral speeds (RTs) in speech CP.

Figure 6:

BrainNet visualization (top to bottom: lateral, medial, and dorsal view) of the brain network (54 edges) identified via stability selection. Color map 1–6 indicates, 1: Frontal (22 ROI), 2: Parietal (10 ROI), 3: Temporal (18 ROI), 4: Occipital (8 ROI), 5: Cingulate (8 ROI), 6: Insula (2 ROI) regions. Node size varies with its degree of connectivity. Connectivity among the same lobe are colored with similar node color. Edge widths represent the weight of absolute correlation (connectivity strength).

Figure 7:

A sparse brain network (8 edges) predicts listeners’ speed (RTs) of speech categorization (57% model accuracy). Red numbers are the ranked importance of the edges describing behavior. Otherwise, as in Figure 6.

Figure 8

Accuracy curves of stability selection (as in Figure 4). Stability selection was applied to Correlation, CH, iCH, PLV PSI based-edge matrix, as well as combinations of CH and iCH, combination of correlation, CH, iCH, and PSI based-edge matrix. Here ‘mul’ and ‘fou’ represents multitaper and Fourier transform methods. The dot point of each accuracy curve indicates maximum accuracy of the optimal combination of features. Correlation-based connectivity outperforms all other measures.

Figure 9:

Brain network underlying Slow RT listeners (left), Medium RT listeners (middle), and Fast RT listeners. Shown here are the most highly correlated (absolute correlation ≥0.5) network edges. Otherwise as in Figs. 6–7. INS, insula; IST, isthmus of cingulate; TRANS, transverse temporal gyrus (auditory cortex); POB, pars orbitalis; PRC, precentral gyrus (motor cortex); PHIP, parahippocampal gyrus; PREC, precunus; l/r, left/right hemisphere.