Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Dec 23;22(1):63.
doi: 10.3390/s22010063.

Deep-Learning-Based Adaptive Advertising with Augmented Reality

Marco A Moreno-Armendáriz  1 , Hiram Calvo  1 , Carlos A Duchanoy  2 , Arturo Lara-Cázares  3 , Enrique Ramos-Diaz  3 , Víctor L Morales-Flores  3
Affiliations

Deep-Learning-Based Adaptive Advertising with Augmented Reality

Marco A Moreno-Armendáriz et al. Sensors (Basel). .

Abstract

In this work we describe a system composed of deep neural networks that analyzes characteristics of customers based on their face (age, gender, and personality), as well as the ambient temperature, with the purpose of generating a personalized signal to potential buyers who pass in front of a beverage establishment; faces are automatically detected, displaying a recommendation using deep learning methods. In order to present suitable digital posters for each person, several technologies were used: Augmented reality, estimation of age, gender, and estimation of personality through the Big Five test applied to an image. The accuracy of each one of these deep neural networks is measured separately to ensure an appropriate precision over 80%. The system has been implemented into a portable solution, and is able to generate a recommendation to one or more people at the same time.

Keywords: augmented reality; computer vision; deep learning; emotion-based recommendation; targeted advertising.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Global system diagram.
Figure 2
Figure 2
Detected face.
Figure 3
Figure 3
Extraction of the faces detected by the SSD from the frames captured by the IP camera, and their processing to be entered into the neural networks.
Figure 4
Figure 4
Diagram showing the flattened input face, the age, and gender network model in TensorRT, and its two outputs: The age obtained from the regressor, and the gender vector obtained from the classifier.
Figure 5
Figure 5
Diagram showing the flattened input face, the classification model: CNN-4 [24] of the Big Five network in TensorRT, and the output vector of size 5 where each position corresponds to a personality dimension as indicated.
Figure 6
Figure 6
Proposed CNN used to estimate age.
Figure 7
Figure 7
Designed transfer learning.
Figure 8
Figure 8
Final architecture of the multitasking convolutional neural network to obtain the age and gender of the person.
Figure 9
Figure 9
CNN-4 classification model architecture for Big Five [24].
Figure 10
Figure 10
Diagram of the recommender operation.
Figure 11
Figure 11
Augmented reality design.
Figure 12
Figure 12
Example of the augmented reality to be carried out.
Figure 13
Figure 13
System hardware connection diagram.
Figure 14
Figure 14
Neural network error Single Shot Detection, note that from epoch 100,000 the error decreases significantly.
Figure 15
Figure 15
Evolution of the error in the training phase of the age estimation.
Figure 16
Figure 16
Evolution of the error in the test phase of the age estimate.
Figure 17
Figure 17
Evolution of precision in the training phase of gender classification.
Figure 18
Figure 18
Evolution of precision in the test phase of gender classification.
Figure 19
Figure 19
Final result of Augmented Reality.
Figure 20
Figure 20
Final result of augmented reality with two people using our advertising totem.
See this image and copyright information in PMC

References

    1. Kwon K., Kim C. How to Design Personalization in a Context of Customer Retention: Who Personalizes What and to What Extent? Electron. Commer. Rec. Appl. 2012;11:101–116. doi: 10.1016/j.elerap.2011.05.002. - DOI
    1. Bleier A., De Keyser A., Verleye K. Customer Engagement through Personalization and Customization. Palgrave Macmillan; Cham, Switzerland: 2018. pp. 75–94.
    1. Das D., Sahoo L., Datta S. A Survey on Recommendation System. Int. J. Comput. Appl. 2017;160:7. doi: 10.5120/ijca2017913081. - DOI
    1. Brusilovski P., Kobsa A., Nejdl W. The Adaptive Web: Methods and Strategies of Web Personalization. Volume 4321 Springer Science & Business Media; Berlin/Heidelberg, Germany: 2007.
    1. AINIA Más del 55 por Ciento de los Millennials Prefiere el Sabor Dulce. [(accessed on 19 December 2020)]. Available online: https://www.ainia.es/noticias/prensa/mas-del-55-por-ciento-de-los-millen....

LinkOut - more resources