Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Oct 14;22(20):7807.
doi: 10.3390/s22207807.

A Practical Approach to the Analysis and Optimization of Neural Networks on Embedded Systems

Mario Merone  1 , Alessandro Graziosi  1 , Valerio Lapadula  1 , Lorenzo Petrosino  1 , Onorato d'Angelis  1 , Luca Vollero  1
Affiliations

A Practical Approach to the Analysis and Optimization of Neural Networks on Embedded Systems

Mario Merone et al. Sensors (Basel). .

Abstract

The exponential increase in internet data poses several challenges to cloud systems and data centers, such as scalability, power overheads, network load, and data security. To overcome these limitations, research is focusing on the development of edge computing systems, i.e., based on a distributed computing model in which data processing occurs as close as possible to where the data are collected. Edge computing, indeed, mitigates the limitations of cloud computing, implementing artificial intelligence algorithms directly on the embedded devices enabling low latency responses without network overhead or high costs, and improving solution scalability. Today, the hardware improvements of the edge devices make them capable of performing, even if with some constraints, complex computations, such as those required by Deep Neural Networks. Nevertheless, to efficiently implement deep learning algorithms on devices with limited computing power, it is necessary to minimize the production time and to quickly identify, deploy, and, if necessary, optimize the best Neural Network solution. This study focuses on developing a universal method to identify and port the best Neural Network on an edge system, valid regardless of the device, Neural Network, and task typology. The method is based on three steps: a trade-off step to obtain the best Neural Network within different solutions under investigation; an optimization step to find the best configurations of parameters under different acceleration techniques; eventually, an explainability step using local interpretable model-agnostic explanations (LIME), which provides a global approach to quantify the goodness of the classifier decision criteria. We evaluated several MobileNets on the Fudan Shangai-Tech dataset to test the proposed approach.

Keywords: Internet of Things; artificial intelligence; convolutional neural network; crowd counting; edge computing; embedded system; optimization method.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
RGB Best trade-off: time inference layer by layer.
Figure 2
Figure 2
Optimization tests over best trade-off. P(i) means Pruning test while R(i) means Regularization tests, both executed on Keras (on blue) and quantized TensorFlow lite (on red) versions. (a) P1–P4 tests. (b) P5, R1–R3 tests.
Figure 3
Figure 3
Interpretability results from over-optimization tests: accuracy and intersection over union trends. (a) P1–P5 tests. (b) R1–R3 tests.
See this image and copyright information in PMC

References

    1. Muhammad G., Hossain M.S. Emotion Recognition for Cognitive Edge Computing Using Deep Learning. IEEE Internet Things J. 2021;8:16894–16901. doi: 10.1109/JIOT.2021.3058587. - DOI
    1. Chen M., Hao Y., Hu L., Hossain M.S., Ghoneim A. Edge-CoCaCo: Toward Joint Optimization of Computation, Caching, and Communication on Edge Cloud. IEEE Wirel. Commun. 2018;25:21–27. doi: 10.1109/MWC.2018.1700308. - DOI
    1. Banbury C.R., Reddi V., Lam M., Fu W., Fazel A., Holleman J., Huang X., Hurtado R., Kanter D., Lokhmotov A., et al. Benchmarking TinyML Systems: Challenges and Direction. arXiv. 20202003.04821
    1. Anguita D., Ghio A., Oneto L., Parra X., Reyes-Ortiz J.L. Human Activity Recognition on Smartphones Using a Multiclass Hardware-Friendly Support Vector Machine. In: Bravo J., Hervás R., Rodríguez M., editors. Ambient Assisted Living and Home Care. Springer; Berlin/Heidelberg, Germany: 2012. pp. 216–223.
    1. Moon S.M., Min M., Nam J., Park J., Lee D., Kim D. Drowsy Driving Warning System Based on GS1 Standards with Machine Learning; Proceedings of the 2017 IEEE International Congress on Big Data (BigData Congress); Honolulu, HI, USA. 25–30 June 2017; pp. 289–296.