Photonic online learning: a perspective

Abstract

Emerging neuromorphic hardware promises to solve certain problems faster and with higher energy efficiency than traditional computing by using physical processes that take place at the device level as the computational primitives in neural networks. While initial results in photonic neuromorphic hardware are very promising, such hardware requires programming or "training" that is often power-hungry and time-consuming. In this article, we examine the online learning paradigm, where the machinery for training is built deeply into the hardware itself. We argue that some form of online learning will be necessary if photonic neuromorphic hardware is to achieve its true potential.

Keywords: integrated photonics; neural networks; neuromorphic photonics.

Figure 1:

Training on photonic hardware ranging from fully offline to fully online. (a) Offline training, where a model of the physical system and the training dataset are trained on a computer, and the weights are transferred to the device. (b) “Fine-tune training”, where the system is trained as in (a), but the weights are adjusted to improve the accuracy after transfer to the device. (c) Hardware-in-the-loop involves measuring the chip during training, but some portion of the calculations for training still happen on a computer. (d) Fully autonomous online learning. In this case, only the training dataset is input to the device, which can adjust its weights autonomously as it learns.

Figure 2:

Proposals for training photonic hardware. (a) and (b) Simulations of fully autonomous online learning proposals for gradient descent via backpropagation in (a) integrated photonics [59] and (b) free space [56]. (c)–(f) Experimentally demonstrated hardware-in-the-loop. (c) The measured activation functions are used in the hardware-aware training of micro-ring resonator based neuromorphic device for optical signal processing [61]. (d) Integrated implementation of DFA where the backward pass is computed on-chip [60]. (e) Fine-tune training of the network increases the accuracy of handwritten digit recognition from 63.9% to 96% [68]. (f) Training of a free-space physical neural network where the calculation of the forward pass in the device prevents accumulation of errors that can happen in offline training [43].

Figure 3:

(a) CIFAR10 testing accuracy versus time during training with SPSA in the MGD framework, as compared to SGD with backpropagation. The time axis has been scaled for MGD operating with 20 kHz weight perturbations, compared to wallclock time for backpropagation performed on a standard desktop GPU. (b) Schematic illustration of perturbing the weights in a microring resonator implementation of photonic neuromorphic hardware. The implementation is shown in a two-layer waveguide process that allows simple waveguide crossings [79] for illustrative purposes; the network can also be realized in a single waveguide layer as in ref. [15].