Sparse Coding Using the Locally Competitive Algorithm on the TrueNorth Neurosynaptic System

Abstract

The Locally Competitive Algorithm (LCA) is a biologically plausible computational architecture for sparse coding, where a signal is represented as a linear combination of elements from an over-complete dictionary. In this paper we map the LCA algorithm on the brain-inspired, IBM TrueNorth Neurosynaptic System. We discuss data structures and representation as well as the architecture of functional processing units that perform non-linear threshold, vector-matrix multiplication. We also present the design of the micro-architectural units that facilitate the implementation of dynamical based iterative algorithms. Experimental results with the LCA algorithm using the limited precision, fixed-point arithmetic on TrueNorth compare favorably with results using floating-point computations on a general purpose computer. The scaling of the LCA algorithm within the constraints of the TrueNorth is also discussed.

Keywords: TrueNorth; brain-inspired; sparse-approximation; sparse-code; sparsity; spiking-neurons.

Figure 1

The corelet used to implement the LCA on the TrueNorth using our novel design methodology.

Figure 2

The summation computation is accurate for every iteration so long as values do not exceed the window size, w = 10.

Figure 3

The left neuron is the positive representation of a variable and the right, the negative of the same variable.

Figure 4

A core with repeated axons and neurons to accommodate positive and negative inputs and outputs.

Figure 5

Restricted precision for vector-matrix multiplication if implemented using synaptic weights directly on the TrueNorth chip.

Figure 6

All layers of our vector-matrix multiply overlaid onto one crossbar array, representing one multiplication matrix element with a binary value of 146. Only the positive representations of inputs and outputs are shown.

Figure 7

Recurrence resulting in incorrect computations of the node state update.

Figure 8

Two subsequent LCA iterations to accommodate recurrence in the system. Triggers enable one path to calculate τ²u[n + 1] over w ticks while the other path sends the prior iteration's values τ²u[n] for use in the calculation.

Figure 9

Programmed core that produces inhibition triggers for the first blocks in both paths of the on-chip dynamic memory processing unit.

Figure 10

Programmed core that sends send and compute triggers to the second blocks in both paths of the on-chip dynamic memory processing unit.

Figure 11

The initial projection repeated on-chip using principles from our on-chip memory corelet.

Figure 12

The output spikes from the LCA corelet on the TrueNorth chip, representing positive and negative representations of τ²u.

Figure 13

The node dynamics of an LCA system with a 33 × 50 dictionary compared to a discrete LCA system. Input signals are y = 14 × Φ₁₆−13 × Φ₃₆ and parameters are τ = 13 and λ = 7.