Integrating unsupervised and reinforcement learning in human categorical perception: A computational model

Abstract

Categorical perception identifies a tuning of human perceptual systems that can occur during the execution of a categorisation task. Despite the fact that experimental studies and computational models suggest that this tuning is influenced by task-independent effects (e.g., based on Hebbian and unsupervised learning, UL) and task-dependent effects (e.g., based on reward signals and reinforcement learning, RL), no model studies the UL/RL interaction during the emergence of categorical perception. Here we have investigated the effects of this interaction, proposing a system-level neuro-inspired computational architecture in which a perceptual component integrates UL and RL processes. The model has been tested with a categorisation task and the results show that a balanced mix of unsupervised and reinforcement learning leads to the emergence of a suitable categorical perception and the best performance in the task. Indeed, an excessive unsupervised learning contribution tends to not identify task-relevant features while an excessive reinforcement learning contribution tends to initially learn slowly and then to reach sub-optimal performance. These results are consistent with the experimental evidence regarding categorical activations of extrastriate cortices in healthy conditions. Finally, the results produced by the two extreme cases of our model can explain the existence of several factors that may lead to sensory alterations in autistic people.

Fig 1

(A): Scheme of the task protocol. The row below shows the examples of inputs that the environment provides to the model (visual input). The middle row shows the trial sequence. Note that a first experimental section occurs before the task experimental section with trials and involves the setting of the task conditions, that is, the choice of the sorting rule and the creation of the ideal responses. The top row zooms in a specific trial, showing the phases that occur during the model-environment interactions. (B): Examples of the 64 geometrical shapes (circles, squares, rectangles, triangles), used to produce the images. Each image encompasses a different attribute out of the four attributes of each of the three categories, namely shape, colour, and size.

Fig 2

(A) A schema of the main model processes involved in its interaction with the environment during the task performance. (B) Scheme of learning processes and targeted brain areas that are addressed by the hypothesis and computational model presented here. The intermediate sensory-motor layers (extrastriate cortices) undergo both associative unsupervised learning (UL) and trial-and-error learning (RL). The latter presents a gradient having a decreasing strength moving from the motor cortex towards the striate cortex.

Fig 3. Schema of the model components and functions, the flows of information between the components, and the learning signals.

Fig 4. A computational schema of the model components and their training algorithms, the flows of information between the components, and the learning signals.

MLP: Multi-layer Perceptron. SLP: Single-layer Perceptron. HL: Hidden Layer. RBM: Restricted Boltzmann Machine. CD: Contrastive Divergence.

Fig 5. Reward per epoch of the five models involving different UL/RL levels, averaged over the models using a given level.

Shaded areas represent the standard deviation.

Fig 6. Performances (maximum reward obtained at the end of training) of models featuring different levels of RL contribution.

Fig 7. Colour sorting category: Reconstructed input.

Principal components of the reconstructed image representations in the case of the colour sorting rule and in correspondence to different levels of RL (shown in different graphs). The dimensionality of the reconstructed image was reduced to two through a PCA (x-axis: first component; y-axis: second component). Within each graph, each reconstructed image is represented by a point marked by an icon that summarises the colour, shape, and size of the shape in the image (some icons are not visible as they overlap). The centroids of the four clusters found by the K-means algorithm are marked with a black dot, while the maximum distance of the points of the cluster from its centroid is shown by a grey circle. A: Level 0 (L0), absent RL (only UL); B: Level 1 (L1), low RL; C: Level 2 (L2), moderate RL; D: Level 3 (L3), high RL; E: Level 4 (L4), extreme RL (no UL).

Fig 8. Shape sorting category: Reconstructed input.

Principal components of the reconstructed image representations in the case of the shape sorting rule and in correspondence to different levels of RL. Note that, in case of overlap, the yellow inputs appear at the top and hide others due to technical factors (we plot the yellow inputs at the end). The plots are drawn as in Fig 7.

Fig 9. Size sorting category: Reconstructed input.

Principal components of the reconstructed image representations in the case of the size sorting rule and in correspondence to different levels of RL. Note that, in case of overlap, the yellow inputs appear at the top and hide others due to technical factors (we plot the yellow inputs at the end). The graphs are drawn as in Fig 7. The red arrow in graph E indicates the centroid of a cluster that contains only the small bars but not the other small shapes.

Fig 10. Information loss for different levels of RL.

Information loss (reconstruction error at the end of the training) of models with different levels of RL.

Fig 11. Input reconstructions (sorting category: Colour) Input reconstructions (sorting category: Shape) Input reconstructions (sorting category: Size).

Image reconstructions with different sorting rules and different levels of RL. A: Original inputs; B: Level 0 (L0)—absent RL (only UL); C: Level 1 (L1)—low RL; D: Level 2 (L2)—moderate RL; E: Level 3 (L3)—high RL; F: Level 4 (L4)—extreme RL (only RL).