A mechanistic account of visual discomfort

Abstract

Much of the neural machinery of the early visual cortex, from the extraction of local orientations to contextual modulations through lateral interactions, is thought to have developed to provide a sparse encoding of contour in natural scenes, allowing the brain to process efficiently most of the visual scenes we are exposed to. Certain visual stimuli, however, cause visual stress, a set of adverse effects ranging from simple discomfort to migraine attacks, and epileptic seizures in the extreme, all phenomena linked with an excessive metabolic demand. The theory of efficient coding suggests a link between excessive metabolic demand and images that deviate from natural statistics. Yet, the mechanisms linking energy demand and image spatial content in discomfort remain elusive. Here, we used theories of visual coding that link image spatial structure and brain activation to characterize the response to images observers reported as uncomfortable in a biologically based neurodynamic model of the early visual cortex that included excitatory and inhibitory layers to implement contextual influences. We found three clear markers of aversive images: a larger overall activation in the model, a less sparse response, and a more unbalanced distribution of activity across spatial orientations. When the ratio of excitation over inhibition was increased in the model, a phenomenon hypothesised to underlie interindividual differences in susceptibility to visual discomfort, the three markers of discomfort progressively shifted toward values typical of the response to uncomfortable stimuli. Overall, these findings propose a unifying mechanistic explanation for why there are differences between images and between observers, suggesting how visual input and idiosyncratic hyperexcitability give rise to abnormal brain responses that result in visual stress.

Keywords: computational modelling; efficient coding; hypermetabolism; interindividual differences; natural scenes; urban scenes; visual discomfort; visual stress.

Figure 1

Schematics of the experiment. (A) We processed images from four sets of stimuli (Architecture 1 and Architecture 2, N = 74 each, Art 1 and Art 2, N = 50 each) using a neurodynamic model of the early visual cortex. This model consisted of a layer of excitatory units and a layer of inhibitory units scattered in a grid of hypercolumns organised retinotopically, each including units sensitive to luminance edges with different spatial orientations and spatial frequencies. The hypercolumns were interconnected through excitatory-excitatory, excitatory-inhibitory, and inhibitory-inhibitory connections following a biologically plausible pattern of lateral connections. For each image, we recorded the firing rates of all the excitatory units in the model over several temporal iterations of the model, leading to (B) the model population response to the image, i.e., vectors of non-negative numbers. (C) Observers reported perceived discomfort when viewing each image in one of the four sets of stimuli by rating each stimulus on a Likert scale in which the lowest value meant ‘not uncomfortable at all’ and the highest ‘very uncomfortable to look at’. (D) We then regressed (here, for illustration, group average of) reported visual discomfort against different metrics of the model population response. The metrics were chosen to reflect three main hypotheses on the neural correlate of visual discomfort (see section Metrics of model activity, Rationale for types of metrics considered). (E) To analyse the contribution to visual discomfort of different spatial frequencies, we also regressed reported visual discomfort against the same metrics applied to the subset of units in the model sensitive to a given spatial frequency (‘frequency channels’, see Methods). The subpanels only show four of the twelve channels.

Figure 2

Illustration of the metrics used as markers of visual discomfort. (A) Activation level of model population response: (from left to right, first panel) Image in the set Architecture 1 with the lowest average activation level, (second) corresponding heatmap of activation summed over all membrane times, frequency channels and orientations, (third) image with the highest average activation level, and (fourth) corresponding heatmap of activation summed as in the second panel; in the heatmaps, the yellower the colour the higher activation. (B) Sparseness of model population response: (first panel) Image with the highest value for the sparseness metric in the same set as in panel (A), (second, black curve) corresponding histogram of firing rates and (grey curve) histogram for the image in the third panel for comparison, (third) image with the lowest value for the sparseness metric, and (fourth, black curve) corresponding histogram of firing rates. (C) Isotropy metric: (first panel) Image with the highest level of the metric in the same set, (second) heatmap of isotropy averaged across all membrane time and frequency channels and (inset) example of distribution of responses across orientations with isotropy equal to the average for the whole image (1.708), (third) Image with the lowest level of the metric, and (fourth) heatmap of isotropy averaged as in panel two and (inset) example of distribution with isotropy equal to the average of the whole image (0.696).

Figure 3

Correlations between average reported visual discomfort against the three main metrics of model population activity, namely (A) model population response activation level, (B) sparseness of the model response, and (C) isotropy in the model response for the four sets of stimuli (from left to right column, Architecture 1, N = 74, Architecture 2, N = 74, Art 1, N = 50, and Art 2, N = 50). Each point represents the Pearson’s correlation coefficient for one frequency channel of the model (dots, for the 12 frequency channels of the model) or for the whole model (triangle). The value of p of each regression is colour-coded with a level of blue (the darker, the lower the value of p), or with yellow for p-values above the reference threshold 0.05. Each inset shows the raw data for the regression in the case of the whole population (triangle). All metrics were normalised to a mean value of 0 and standard deviation of 0.5 (see Methods).

Figure 4

Comparison of the distribution of isotropy in each set with that in nature and regression of isotropy against average reported discomfort for (A) the set Architecture 1, (B) Architecture 2, (C) Art 1, and (D) Art 2. In each panel, the left plot shows the distribution of isotropy in the model for all stimuli in the set (grey, left of the central axis) and distribution for a set of natural images (green, right of central axis). The dots show the raw values of the metric for each stimulus (Architecture 1, N = 74, Architecture 2, N = 74, Art 1, N = 50, Art 2, N = 50, and natural images, N = 100), the box plot show the first (bottom) and third (top) quartile, the notches show the 95% confidence interval for the median, and the star and circle show the mean of the distributions. The right plots in each panel show the regression of isotropy against average reported discomfort (grey) as well as the mean value of the metric for the set of natural images.

Figure 5

Changes in markers of visual discomfort when the balance of excitation over inhibition is modified. Distributions of (A) activation, (B) sparseness, and (C) isotropy metrics for all the stimuli in Architecture 1 and increasing values of gain for the inhibitory layer. The gain ranged from 0, i.e., no inhibitory activity in the model (top left, light grey distribution), to 1, i.e., reference model (top right, blue distribution), in steps of 0.125. Differences between distributions and the distribution for the reference model were tested using two-sample Kolmogorov–Smirnov tests; p-values are colour coded as in Figure 3. See Supplementary Figures S11–S13 for the equivalent distributions for sets Architecture 2, Art 1 and 2.