Autoshaping and automaintenance: a neural-network approach

Abstract

This article presents an interpretation of autoshaping, and positive and negative automaintenance, based on a neural-network model. The model makes no distinction between operant and respondent learning mechanisms, and takes into account knowledge of hippocampal and dopaminergic systems. Four simulations were run, each one using an A-B-A design and four instances of feedfoward architectures. In A, networks received a positive contingency between inputs that simulated a conditioned stimulus (CS) and an input that simulated an unconditioned stimulus (US). Responding was simulated as an output activation that was neither elicited by nor required for the US. B was an omission-training procedure. Response directedness was defined as sensory feedback from responding, simulated as a dependence of other inputs on responding. In Simulation 1, the phenomena were simulated with a fully connected architecture and maximally intense response feedback. The other simulations used a partially connected architecture without competition between CS and response feedback. In Simulation 2, a maximally intense feedback resulted in substantial autoshaping and automaintenance. In Simulation 3, eliminating response feedback interfered substantially with autoshaping and automaintenance. In Simulation 4, intermediate autoshaping and automaintenance resulted from an intermediate response feedback. Implications for the operant-respondent distinction and the behavior-neuroscience relation are discussed.

Fig 1. A typical network architecture.

Units labeled as I₁, I₂, and I₃ are input units whose activations represent the kinds of exteroceptive stimuli used in respondent conditioning as CSs (e.g., light and tones). Reinforcement (US occurrence) is represented by an activation of the input unit labeled as I₄. The dashed arrows labeled as “CS”, “R feedback”, and “US” represent the input activations that defined the stimuli used in Simulation 1. Thin black arrows represent variable connections whose weights changed according to a learning function. Thick black arrows represent maximally strong unmodifiable connections. Gray arrows and areas represent the signals that influence weight changes. Responding is represented by the activation of the output units, labeled as R and CR/UR. The other labels denote: sa: sensory-association; ma: motor-association; ca1: Cornu Ammon 1; vta: ventral tegmental area.

Fig 2. Results of Simulation 1.

The rows represent the individual networks (labeled as N1, N2, N3, and N4). Each network was given a sequence of three phases, represented by the columns labeled as AS/PAM (autoshaping/positive automaintenance), NAM (negative automaintenance), and PAM. The dots represent R activations at ts  =  7 (the moment before reinforcement). The dashed lines mark the R activation criterion for sensory feedback from responding.

Fig 3. Network architecture used in Simulations 2, 3, and 4.

Labels I₁ through I₅ represent the exteroceptive sensory input units. The dashed arrows labeled as “CS” and “R feedback” represent the activations that defined the stimuli used in the simulations. The US and CR/UR units, as well as diffuse signals (shown in Figure 1), were omitted for simplicity. Signals influencing weight changes were averaged across ca1 or vta units.

Fig 4. Results of Simulation 2.

The rows represent the individual networks (labeled as N5, N6, N7, and N8). Each network was given a sequence of three phases, represented by the columns labeled as AS/PAM (autoshaping/positive automaintenance), NAM (negative automaintenance), and PAM. The dots represent R activations at ts  =  7 (the moment before reinforcement). The dashed lines mark the R activation criterion for sensory feedback from responding.

Fig 5. Results of Simulation 3.

The rows represent the individual networks (labeled as N9, N10, N11, and N12). Each network was given a sequence of three phases, represented by the columns labeled as AS/PAM (autoshaping/positive automaintenance), NAM (negative automaintenance), and PAM. The dots represent R activations at ts  =  7 (the moment before reinforcement). In this simulation, R activations had no sensory feedback. Hence, dashed lines are not shown, and the dashed arrows labeled as “R feedback” in Figure 3 do not apply here.

Fig 6. Results of Simulation 4.

The rows represent the individual networks (labeled as N13, N14, N15, and N16). Each network was given a sequence of three phases, represented by the columns labeled as AS/PAM (autoshaping/positive automaintenance), NAM (negative automaintenance), and PAM. The dots represent R activations at ts  =  7 (the moment before reinforcement). The dashed lines mark the R activation criterion for sensory feedback from responding.