Human visual consciousness involves large scale cortical and subcortical networks independent of task report and eye movement activity

Abstract

The full neural circuits of conscious perception remain unknown. Using a visual perception task, we directly recorded a subcortical thalamic awareness potential (TAP). We also developed a unique paradigm to classify perceived versus not perceived stimuli using eye measurements to remove confounding signals related to reporting on conscious experiences. Using fMRI, we discovered three major brain networks driving conscious visual perception independent of report: first, increases in signal detection regions in visual, fusiform cortex, and frontal eye fields; and in arousal/salience networks involving midbrain, thalamus, nucleus accumbens, anterior cingulate, and anterior insula; second, increases in frontoparietal attention and executive control networks and in the cerebellum; finally, decreases in the default mode network. These results were largely maintained after excluding eye movement-based fMRI changes. Our findings provide evidence that the neurophysiology of consciousness is complex even without overt report, involving multiple cortical and subcortical networks overlapping in space and time.

Fig. 1. Report-dependent and report-independent behavioral paradigms for conscious perception; similar pupillometry and eye tracking responses.

A Report Paradigm trial sequence, stimulus locations, and opacity conditions. B Report + No-Report Paradigm trial sequence, stimulus locations, and opacity conditions. C Report (task-relevant) data with perception based on overt participant responses and D no-report (task-irrelevant) data with perception classified by machine learning yield similar pupil, blink, and microsaccade dynamics (Full details on methods and materials are available in the Supplementary Information). Data in C and D are from stimuli at perceptual threshold (see B inset) in the Report + No-Report Paradigm (N = 68 participants). Stimulus onset was at time = 0 seconds (s). Classification score threshold for D was 0 (Fig. S9). Statistically significant (Statistically Sig.) different time points between perceived and not perceived data are indicated (black bars above pupil, blink, and microsaccade timecourses) based on cluster-based permutation tests (p < 0.05) (Full details on methods and materials are available in the Supplementary Information). Milliseconds (ms). Standard error of the mean (SEM). The face stimulus (A, B) is sourced from the FACES database (Ebner, N. C., Riediger, M., & Lindenberger, U. (2010). FACES—A database of facial expressions in young, middle-aged, and older women and men: Development and validation. Behavior Research Methods, 42, 351–362. doi:10.3758/BRM.42.1.351).

Fig. 2. Cortical and thalamic electrophysiology signals in conscious perception.

A–C Scalp topographical plots of high-density scalp EEG showing mean voltage (microvolts; µV) for all statistically significant samples by cluster-based permutation tests (p < 0.05) in four time windows corresponding with the event-related potentials (ERPs) N100 (75-125 milliseconds; ms), visual awareness negativity (VAN; 175–225 ms), P2/N2 (275–325 ms), and P3 (350–650 ms) for perceived versus not perceived stimuli in A report, B no-report, and C report versus no-report data. The VAN is present both for report and no-report data, whereas later ERPs, especially N2 and P3 are report-dependent. D Subcortical depth channel locations collapsed onto a coronal slice (−19.9 mm) in MNI brain template space. Red/purple (N = 13) and white bipolar channels (N = 11) distinguish above or below voltage threshold channels, respectively (Fig. S12) (Full details on methods and materials are available in the Supplementary Information). Red channels are with the RNS System (NeuroPace, Inc.) and purple channels with Natus NeuroWorks (Natus, Inc.) depth electrode recordings (Table S2). Neighboring anatomy includes lateral ventricles (LV), third ventricle (3rd V), thalamus (Th), midbrain tegmentum (MT), and pons. Right cortical hemisphere (R) and left cortical hemisphere (L). E Thalamic awareness potential (TAP) and P3 are seen, respectively in thalamic above-threshold contacts and Pz (location on inset) scalp EEG contacts. Mean timecourses show significant differences for perceived versus not perceived stimuli in the ERP analysis for TAP and P3 by cluster-based permutation tests (*p < 0.05) (Full details on methods and materials are available in the Supplementary Information). Stimulus onset was at time = 0 ms. F Peak latencies from stimulus onset for scalp ERPs and TAP. Circles represent individual data (mean channel latencies) and error bars indicate standard error of the mean (SEM). Significantly different latencies from TAP found by two-sided Wilcoxon rank sum test with Holm–Bonferroni correction (*p < 0.05; N100 versus TAP p = 0.014; VAN versus TAP p = 0.038; N2 versus TAP p = 0.014; P3 versus TAP p = 0.040; all p values are Holm-Bonferroni corrected). A–C Data for report stimuli are from Report Paradigm (N = 57) and Report + No-Report Paradigm (N = 65); data for no-report stimuli are from Report + No-Report Paradigm (N = 65). D–F Data are from Report Paradigm in patient participants with thalamic depth electrodes (N = 7).

Fig. 3. Whole brain fMRI maps for conscious perception with and without overt report.

A, B Report and C, D no-report perceived minus not perceived statistically significant voxels from cluster-based permutation tests (p < 0.05) at 3 and 6 seconds (s) post-stimulus presentation (Full details on methods and materials are available in the Supplementary Information). Statistically significant positive and negative voxel cluster t-values are shown in warm and cool colors, respectively. Anatomical regions are labeled at their peak response between the 3 and 6-s time points. E, F Voxel-level conjunction analysis of report and no-report, perceived minus not perceived statistical whole brain maps with shared (report-independent) increases shown in green and shared decreases shown in purple. Pontine tegmentum (PT), midbrain tegmentum (MT), thalamus (Th), nucleus accumbens (NA), striatum (Str), lateral superior cerebellum (Crus I/II), anterior insula/claustrum (AI), anterior cingulate (AC), supplementary motor area (SMA), primary visual cortex (V1), fusiform gyrus (FG), anterior middle frontal gyrus (AMFG), posterior middle frontal gyrus (PMFG), frontal eye fields (FEF), frontal pole (FP), orbital frontal cortex (OFC), ventral medial prefrontal cortex (VMFC), anterior inferior parietal lobule (AIPL), dorsal inferior parietal lobule (DIPL), posterior inferior parietal lobule (PIPL), superior parietal lobule (SPL), medial parietal cortex (MP), posterior cingulate/precuneus (PC), and anterolateral temporal cortex (ALT). Right cortical hemisphere (R) and left cortical hemisphere (L). See Slides S1, 2, and 4 for all times 20 s pre and post-stimulus presentation for report, no-report, and conjunction results. Data for report stimuli are from the Report Paradigm (N = 34) and Report + No-Report Paradigm (N = 65); data for no-report stimuli are from the Report + No-Report Paradigm (N = 65). Results are visualized on the brain volume PD25-MPRAGET1w-template-500um.nii (0.5 mm³ voxels) from an MNI registration with the BigBrain dataset (https://osf.io/xkqb3/)^,.

Fig. 4. Whole brain fMRI surface maps for conscious perception with and without overt report.

The same results presented in Fig. 3 plotted on an inflated MNI ICBM152 anatomical brain surface mesh. A, B Report and C, D no-report perceived minus not perceived statistically significant voxels from cluster-based permutation tests (p < 0.05) at 3 and 6 seconds (s) post-stimulus presentation (Full details on methods and materials are available in the Supplementary Information). Statistically significant positive and negative voxel cluster t-values are shown in warm and cool colors, respectively. Anatomical regions are labeled at their peak response between the 3 and 6-s time points. E, F Voxel-level conjunction analysis of report and no-report, perceived minus not perceived statistical whole brain maps with shared (report-independent) increases shown in green and shared decreases shown in purple. Midbrain tegmentum (MT), thalamus (Th), anterior insula/claustrum (AI), anterior cingulate (AC), supplementary motor area (SMA), primary visual cortex (V1), fusiform gyrus (FG), anterior middle frontal gyrus (AMFG), posterior middle frontal gyrus (PMFG), frontal eye fields (FEF), frontal pole (FP), orbital frontal cortex (OFC), ventral medial prefrontal cortex (VMFC), anterior inferior parietal lobule (AIPL), dorsal inferior parietal lobule (DIPL), posterior inferior parietal lobule (PIPL), superior parietal lobule (SPL), medial parietal cortex (MP), posterior cingulate/precuneus (PC), and anterolateral temporal cortex (ALT). See Fig. 3 for additional changes in subcortical structures not visible in surface renderings (midbrain tegmentum, thalamus, pontine tegmentum, striatum, nucleus accumbens, cerebellar Crus I/II, cerebellar vermis). Right cortical hemisphere (R) and left cortical hemisphere (L). Data for report stimuli are from the Report Paradigm (N = 34) and Report + No-Report Paradigm (N = 65); data for no-report stimuli are from the Report + No-Report Paradigm (N = 65). Results are visualized on the brain surface mesh image BrainMesh_ICBM152_smoothed from Surf Ice (Version 12.1; https://www.nitrc.org/projects/surfice/).

Fig. 5. Whole brain fMRI maps for interstimulus interval (ISI) classified data and for eye-movement comparisons with no-report data.

A, B ISI classified “perceived” minus “not perceived” and C, D no-report perceived minus not perceived versus ISI classified “perceived” minus “not perceived” data. Statistically significant voxels are shown from cluster-based permutation tests (p < 0.05) at 3 and 6 seconds (s) post-stimulus presentation for no-report data or post the ISI center time (Full details on methods and materials are available in the Supplementary Information). Statistically significant positive and negative voxel cluster t-values are shown in warm and cool colors, respectively. Anatomical regions are labeled at their peak response between the 3 and 6-second time points. E, F No-report perceived minus not perceived data with removal of pupil, blink, and microsaccade-related fMRI signals by voxel-level conjunction analysis (Full details on methods and materials are available in the Supplementary Information). Whole brain maps with increases shown in green and decreases shown in purple. Compare these results with Fig. 3E, F. Pontine tegmentum (PT), midbrain tegmentum (MT), thalamus (Th), nucleus accumbens (NA), striatum (Str), lateral superior cerebellum (Crus I/II), anterior insula/claustrum (AI), anterior cingulate (AC), primary visual cortex (V1), fusiform gyrus (FG), anterior middle frontal gyrus (AMFG), posterior middle frontal gyrus (PMFG), frontal eye fields (FEF), frontal pole (FP), ventral medial prefrontal cortex (VMFC), anterior inferior parietal lobule (AIPL), dorsal inferior parietal lobule (DIPL), posterior inferior parietal lobule (PIPL), superior parietal lobule (SPL), medial parietal cortex (MP), posterior cingulate/precuneus (PC), and anterolateral temporal cortex (ALT). Right cortical hemisphere (R) and left cortical hemisphere (L). Data are from the Report + No-Report Paradigm (N = 65). Results are visualized on the brain volume PD25-MPRAGET1w-template-500 μm.nii (0.5 mm³ voxels) from an MNI registration with the BigBrain dataset (https://osf.io/xkqb3/)^,.

Fig. 6. Data-driven anatomical clustering of fMRI signals for conscious perception and region of interest (ROI) timecourses.

A–C Three large-scale networks found with k-means clustering based on fMRI timecourses (Fig. S20). A Detection, arousal, and salience networks (DAS), B task-positive networks (TPN), and C default mode network (DMN). Cortical and subcortical/cerebellar territories are shown in different shades. D–F Mean percent change blood-oxygen-level-dependent (BOLD) timecourses for report (green) and no-report (purple) perceived minus not perceived conditions for DAS, TPN, and DMN. G–J Selected subregion mean percent change BOLD timecourses (see also Figs. S21–23). Percentages of voxels from a particular network (DAS, TPN, or DMN) versus all network voxels found in each ROI are shown for report (green) and no-report (purple) data (Full details on methods and materials are available in the Supplementary Information). K Anatomical visualization of subregion ROIs from G, H, I, and J. Pontine tegmentum (PT), midbrain tegmentum (MT), thalamus (Th), nucleus accumbens (NA), striatum (Str), anterior insula/claustrum (AI), anterior cingulate (AC), supplementary motor area (SMA), primary visual cortex (V1), fusiform gyrus (FG), anterior middle frontal gyrus (AMFG), posterior middle frontal gyrus (PMFG), frontal eye fields (FEF), frontal pole (FP), orbital frontal cortex (OFC), ventral medial prefrontal cortex (VMFC), anterior inferior parietal lobule (AIPL), dorsal inferior parietal lobule (DIPL), posterior inferior parietal lobule (PIPL), superior parietal lobule (SPL), medial parietal cortex (MP), posterior cingulate/precuneus (PC), and anterolateral temporal cortex (ALT). Right cortical hemisphere (R) and left cortical hemisphere (L). Stimulus onset was at time = 0 seconds (s). Data for report stimuli are from the Report Paradigm (N = 34) and Report + No-Report Paradigm (N = 65); data for no-report stimuli are from the Report + No-Report Paradigm (N = 65). Clusters (A-C) and ROIs (K) are visualized on the brain volume PD25-MPRAGET1w-template-500um.nii (0.5 mm³ voxels) from an MNI registration with the BigBrain dataset (https://osf.io/xkqb3/)^,.