Abrupt and spontaneous strategy switches emerge in simple regularised neural networks

Abstract

Humans sometimes have an insight that leads to a sudden and drastic performance improvement on the task they are working on. Sudden strategy adaptations are often linked to insights, considered to be a unique aspect of human cognition tied to complex processes such as creativity or meta-cognitive reasoning. Here, we take a learning perspective and ask whether insight-like behaviour can occur in simple artificial neural networks, even when the models only learn to form input-output associations through gradual gradient descent. We compared learning dynamics in humans and regularised neural networks in a perceptual decision task that included a hidden regularity to solve the task more efficiently. Our results show that only some humans discover this regularity, and that behaviour is marked by a sudden and abrupt strategy switch that reflects an aha-moment. Notably, we find that simple neural networks with a gradual learning rule and a constant learning rate closely mimicked behavioural characteristics of human insight-like switches, exhibiting delay of insight, suddenness and selective occurrence in only some networks. Analyses of network architectures and learning dynamics revealed that insight-like behaviour crucially depended on a regularised gating mechanism and noise added to gradient updates, which allowed the networks to accumulate "silent knowledge" that is initially suppressed by regularised gating. This suggests that insight-like behaviour can arise from gradual learning in simple neural networks, where it reflects the combined influences of noise, gating and regularisation. These results have potential implications for more complex systems, such as the brain, and guide the way for future insight research.

Fig 1

Stimuli, task design and insight classification procedure (A) Stimuli and stimulus-response mapping: dot clouds were either coloured in orange or purple and moved to one of the four directions NW, NE, SE, SW with varying coherence. A left response key, “X”, corresponded to the NW/SE motion directions, while a right response key “M” corresponded to NE/SW directions. (B) Task structure of the two-alternative forced choice task for humans and neural networks: each block consisted of 100 trials. A first training block only for humans contained only 100% motion coherence trials to familiarise subjects with the S-R mapping. The remaining training blocks contained only high coherence (0.2, 0.3, 0.45) trials. In the motion phase, colour changed randomly and was not predictive and all motion coherence levels were included. Colour was predictive of correct choices and correlated with motion directions as well as correct response buttons only in the last five blocks (motion and colour phase). Participants were instructed to use colour before the very last block 9, which served as sanity check (data shown only in SI). (C) Trial structure: a fixation cue is shown for a duration that is shuffled between 400, 600, 800 and 1000 ms. The random dot cloud stimulus is displayed for 2000 ms. A response can be made during these entire 2000 ms, but a central feedback cue will replace the fixation cue immediately after a response. (D) Schematic of the neural network with regularised gate modulation used to model insights. (E) Insight classification procedure: We fitted a sigmoid model ($y=\frac{y_{max}-y_{min}}{1+e^{-m(t-t_s)}}+y_{min}$, see Methods for details) to data from the lowest motion coherence condition data of both the Experimental Group and a Control Group (where colour never becomes predictive), and derived the distributions of the slope steepness at the estimated switch point (inflection point t_s of sigmoid function). We then asked which participants from the Experimental group had fitted slopes that were steeper than the 100th percentile of the Control Group. The resulting purely behavioural classification of insights agreed to 79.6% with verbal insight reports from a post-task questionnaire and predicted a number of behavioural features (see text). Importantly, using this method allowed us to apply the same procedure to neural networks.

Fig 2

Humans: task performance and insight-like strategy switches (A) Accuracy (% correct) during the motion phase increases with increasing motion coherence. N = 99, error bars signify standard error of the mean (SEM). (B) Accuracy (% correct) over the course of the experiment for all motion coherence levels. First dashed vertical line marks the onset of the colour predictiveness (motion and colour phase), second dashed vertical line the “instruction” about colour predictiveness. Blocks shown are halved task blocks (50 trials each). N = 99, error shadows signify SEM. (C) Switch point-aligned accuracy on lowest motion coherence level for insight (49/99) and no-insight (50/99) subjects. Blocks shown are halved task blocks (50 trials each). Error shadow signifies SEM. (D) Trial-wise switch-aligned smoothed binary responses on lowest motion coherence level for an example insight subject. (E) Illustration of the sigmoid function for different slope steepness parameters. (F) Difference between BICs of the linear and sigmoid function for each human subject. N = 99. (G) Distributions of fitted slope steepness at inflection point parameter for control experiment and classified insight and no-insight groups. (H) Distribution of switch points. Dashed vertical line marks onset of colour predictiveness. Blocks shown are halved task blocks (50 trials each).

Fig 3

L1-regularised neural networks: task performance and insight-like strategy switches (A) Accuracy (% correct) during the motion phase increases with increasing motion coherence. N = 99, error bars signify SEM. Grey line is human data for comparison. (B) Accuracy (% correct) over the course of the experiment for all motion coherence levels. First dashed vertical line marks the onset of the colour predictiveness (motion and colour phase), second dashed vertical line the “instruction” about colour predictiveness. Blocks shown are halved task blocks (50 trials each). N = 99, error shadows signify SEM. (C) Switch point-aligned accuracy on lowest motion coherence level for insight (46/99) and no-insight (53/99) networks. Blocks shown are halved task blocks (50 trials each). Error shadow signifies SEM. (D) Trial-wise switch-aligned continuous outputs on lowest motion coherence level for an example insight network. (E) Switch point-aligned accuracy on lowest motion coherence level for insight (18/99) and no-insight (81/99) hidden layer networks. Blocks shown are halved task blocks (50 trials each). Error shadow signifies SEM. (F) Difference between BICs of the linear model and sigmoid function for each network. (G) Distributions of fitted slope steepness at inflection point parameter for control networks and classified insight and no-insight groups. (H) Distribution of switch points. Dashed vertical line marks onset of colour predictiveness. Blocks shown are halved task blocks (50 trials each).

Fig 4. Differences between insight and no-insight networks in parameter dynamics.

Absolute average magnitude of motion (light purple) and colour (dark purple) gates at the first trial (A) and the last trial (B) of the motion and colour phase, shown separately for insight and no-insight networks. Differences between the gates of insight vs no-insight networks emerged only at the end of motion and colour phase, i.e. only after insight-like behaviour had occurred. Panels (C) and (D) show the same for absolute weight magnitudes. Unlike gates, colour weights differed between network types already before insight-like switches were detectable, i.e. at the start of the motion and colour phase. Both absolute colour weight (E) and gate (F) magnitudes increase after start of the motion and colour phase (trial 200, dashed vertical line) for insight networks, while absolute colour gates decrease in magnitude after colour correlation onset. For no-insight networks absolute weight (G) and gate (H) magnitudes do not show such dynamics and remain stable after the onset of the motion and colour phase. Error bars/shadows signify SEM.

Fig 5

Influence of gradient noise σ_ξ and regularisation λ on insight-like switches (A) Influence of gradient noise (standard deviation σ_ξ) on the frequency of absolute numbers of insight-like networks. The frequency of insight-like switches increases gradually with σ_ξ until it reaches a ceiling around σ_ξ = .05. Error bars are SD. Results from 10 simulations of 99 networks each. (B) Effects of gradient noise added only to either all weights (${\sigma }_{{\xi }_w}$, light purple dashed line), all gates (${\sigma }_{{\xi }_g}$, solid purple line), all motion parameters (i.e. motion weight and motion gates, ${\sigma }_{{\xi }_{g_m,w_m}}$, light purple solid line) and all colour parameters (${\sigma }_{{\xi }_{g_c,w_c}}$, dark purple solid line) on the frequency of insight-like switches. While only small amounts of noise in the colour parameters ${\sigma }_{{\xi }_{g_c,w_c}}$ suffice to induce 100% insight-like strategy switches among networks (dark purple line), adding noise to the motion parameters ${\sigma }_{{\xi }_{g_c,w_c}}$ (light purple solid line) only had a a very minor effect. Furthermore, we find that adding noise specifically to the weights (${\sigma }_{{\xi }_w}$, dashed purple line), has a much larger effect than adding noise to the gates (${\sigma }_{{\xi }_g}$, solid purple line). Error bars are SD. Results from 10 simulations of 99 networks each. Colour scheme as in Fig 1B (C) The frequency of insight-like switches declines with increasing λ. (D) The average switch point occurs later in the task with increasing λ. Error bars signify SEM.