Modeling multisensory enhancement with self-organizing maps

Abstract

Self-organization, a process by which the internal organization of a system changes without supervision, has been proposed as a possible basis for multisensory enhancement (MSE) in the superior colliculus (Anastasio and Patton, 2003). We simplify and extend these results by presenting a simulation using traditional self-organizing maps, intended to understand and simulate MSE as it may generally occur throughout the central nervous system. This simulation of MSE: (1) uses a standard unsupervised competitive learning algorithm, (2) learns from artificially generated activation levels corresponding to driven and spontaneous stimuli from separate and combined input channels, (3) uses a sigmoidal transfer function to generate quantifiable responses to separate inputs, (4) enhances the responses when those same inputs are combined, (5) obeys the inverse effectiveness principle of multisensory integration, and (6) can topographically congregate MSE in a manner similar to that seen in cortex. Thus, the model provides a useful method for evaluating and simulating the development of enhanced interactions between responses to different sensory modalities.

Keywords: artificial neural networks; competitive learning; computational modeling; multisensory integration; self-organization; superior colliculus.

Figure 1

A sigmoidal enhancement example. This curve simulates the physiological relationship between level of input to an artificial neuron (dot product) and its hypothetical output (sigmoid). By selecting points along this input–output relationship, a value for sigmoid enhancement can be calculated. In this example, SM 1 and SM 2 correspond to unimodal responses, CM to the bimodal response, and the dot product corresponds to the value of the weighted sum of inputs. The sigmoidal enhancement in this example is (0.9 − 0.3)/0.3 = 200%.

Figure 2

Activation and multisensory enhancement (MSE) levels for a three modality model on a 10 × 10 sized SOM after 5,000 iterations of training; throughout which, the neighborhood width was kept constant at 1. Activation maps in the top row (labeled “unimodal”) show distinct activation areas (yellow) for single-modality stimulation of the different modalities (001, 010, 100). The second row shows activation levels for combined-modality stimulation (011, resulting from combined inputs from 001 and 010; 101 resulting from combining 001 and 100; and 110 resulting from combining 010 and 100). Here, the highest levels of activity (red, dark red) occurred in a single region between the representations of the constituent unimodal inputs. In row three, the level of multisensory enhancement is determined for the 011, 101, and 110 stimulus combinations (see Section “Materials and Methods”), and reveals a sharper focus of multisensory enhancement levels at the locus between the unimodal representations. The bottom row illustrates an activation map for stimulation in all three modalities (111) and the resulting levels of multisensory enhancement (MSE 111). The map in the box depicts the result of spontaneous activity (000) without driving, wherein only low levels of activity resulted across the map. Scale bars on right indicate activation and MSE levels.

Figure 3

Quantification of multisensory enhancement for a two modality model. Activation maps in the top row show unimodal activation areas (01, 10) and the result of bimodal stimulation (11), from which the data in the middle row are derived. For a given map, representative levels of activity was measured from neurons lying on a diagonal line from the top left to bottom right and these response values are depicted by the blue, curved line (second row). Note that unimodal responses (01, 10) were distributed toward the edges of the map, while bimodal responses (11) were centered between them. In each condition, bimodal responses exceeded that produced by unimodal stimulation (histograms, bottom row), but the greatest levels of activation, representing multisensory enhancement, were achieved at the position (5,5) between the unimodal areas of the map (third row). Weight vectors for the trained SOM neurons at positions (1,1), (5,5), and (10,10) are depicted in the last row of the figure.

Figure 4

A two modality SOM showing inverse effectiveness. This 3D graph plots the position of artificial neurons in the trained SOM (location 1,1 10,10; x-axis) against the driven value of the inputs (0.4–1.0; y-axis) and the level of multisensory enhancement (MSE; z-axis). Note that the same positions (e.g., neuron 6,6) on the SOM had different levels of MSE depending on the driven value of the input. If the driven value was low (0.4–0.5), the maximal level of MSE generated was high (>400%); but if the driven value was high (0.9–1.0), the maximal MSE level was low (<60%). This inverse relationship between driven value (stimulus strength or effectiveness) and MSE is similar to that observed for biological multisensory neurons.