SNN4Agents: a framework for developing energy-efficient embodied spiking neural networks for autonomous agents

Abstract

Recent trends have shown that autonomous agents, such as Autonomous Ground Vehicles (AGVs), Unmanned Aerial Vehicles (UAVs), and mobile robots, effectively improve human productivity in solving diverse tasks. However, since these agents are typically powered by portable batteries, they require extremely low power/energy consumption to operate in a long lifespan. To solve this challenge, neuromorphic computing has emerged as a promising solution, where bio-inspired Spiking Neural Networks (SNNs) use spikes from event-based cameras or data conversion pre-processing to perform sparse computations efficiently. However, the studies of SNN deployments for autonomous agents are still at an early stage. Hence, the optimization stages for enabling efficient embodied SNN deployments for autonomous agents have not been defined systematically. Toward this, we propose a novel framework called SNN4Agents that consists of a set of optimization techniques for designing energy-efficient embodied SNNs targeting autonomous agent applications. Our SNN4Agents employs weight quantization, timestep reduction, and attention window reduction to jointly improve the energy efficiency, reduce the memory footprint, optimize the processing latency, while maintaining high accuracy. In the evaluation, we investigate use cases of event-based car recognition, and explore the trade-offs among accuracy, latency, memory, and energy consumption. The experimental results show that our proposed framework can maintain high accuracy (i.e., 84.12% accuracy) with 68.75% memory saving, 3.58x speed-up, and 4.03x energy efficiency improvement as compared to the state-of-the-art work for the NCARS dataset. In this manner, our SNN4Agents framework paves the way toward enabling energy-efficient embodied SNN deployments for autonomous agents.

Keywords: automotive data; autonomous agents; energy efficiency; neuromorphic computing; neuromorphic processor; spiking neural networks.

FIGURE 1

Experimental results for observing the impact of different weight precision levels (i.e., 32, 12, 8, and 4 bits) on: (A) the accuracy of an SNN model across the training epochs; and (B) the accuracy after 200 training epoch and the corresponding memory footprints.

FIGURE 2

The overview of our novel contributions, shown in green boxes.

FIGURE 3

Overview of the functionality of a Spiking Neural Network.

FIGURE 4

The flow of (A) PTQ, Post-Training Quantization, and (B) QAT, Quantization-Aware Training.

FIGURE 5

Illustration of different rounding schemes: Truncation (TR), Rounding-to-the-Nearest (RN), and Stochastic Rounding (SR); based on studies in (Putra and Shafique, 2021a).

FIGURE 6

The flow of our SNN4Agents framework, with the technical contributions highlighted in green.

FIGURE 7

Results of accuracy across different precision levels (i.e., 32, 16, 12, 10, 8, and 4 bit).

FIGURE 8

Results of accuracy across different timestep settings (i.e., 20, 15, 10, and 5), while considering different precision levels (i.e., 32, 16, and 4 bit). Here, $B$ b_ $T$ t denotes $B$ bit precision and $T$ timestep.

FIGURE 9

Results of accuracy across different attention window sizes (i.e., $100\times 100$ and $50\times 50$ ).

FIGURE 10

Experimental setup for evaluating our SNN4Agents framework. The proposed settings from our SNN4Agents are incorporated into the experimental setup as highlighted in green.

FIGURE 11

Experimental results for accuracy across different parameter settings, including precision levels, timesteps, and training epochs under (A) $100\times 100$ attention window, and (B) $50\times 50$ attention window.

FIGURE 12

Experimental results for memory footprint normalized to the baseline (32b_100w) across different precision levels and attention window sizes (i.e., $100\times 100$ and $50\times 50$ ).

FIGURE 13

Experimental results for latency normalized to the baseline (32b_20t) across different parameter settings, including precision levels, timesteps, and training epochs under (A) $100\times 100$ attention window, and (B) $50\times 50$ attention window.

FIGURE 14

Experimental results for energy consumption normalized to the baseline (32b_20t) across different parameter settings, including precision levels, timesteps, and training epochs under (A) $100\times 100$ attention window, and (B) $50\times 50$ attention window.

FIGURE 15

(A) Trade-off analysis for accuracy vs memory, accuracy vs normalized latency, and accuracy vs normalized energy consumption. (B) Computational complexity of different designs across different attention window sizes (i.e., $100\times 100$ and $50\times 50$ ) and timestep settings (i.e., 20, 15, 10, and 5).