A Real-Time Application for the Analysis of Multi-Purpose Vending Machines with Machine Learning

Abstract

With the development of mobile payment, the Internet of Things (IoT) and artificial intelligence (AI), smart vending machines, as a kind of unmanned retail, are moving towards a new future. However, the scarcity of data in vending machine scenarios is not conducive to the development of its unmanned services. This paper focuses on using machine learning on small data to detect the placement of the spiral rack indicated by the end of the spiral rack, which is the most crucial factor in causing a product potentially to get stuck in vending machines during the dispensation. To this end, we propose a k-means clustering-based method for splitting small data that is unevenly distributed both in number and in features due to real-world constraints and design a remarkably lightweight convolutional neural network (CNN) as a classifier model for the benefit of real-time application. Our proposal of data splitting along with the CNN is visually interpreted to be effective in that the trained model is robust enough to be unaffected by changes in products and reaches an accuracy of 100%. We also design a single-board computer-based handheld device and implement the trained model to demonstrate the feasibility of a real-time application.

Keywords: convolutional neural network; k-means; real-time application; small data; smart vending machines; visual interpretation.

Figure 1

(a) An overall view of a multi-purpose vending machine in which most racks are spirals. The mechanism of dispensing products by the spiral rack with (b) a potential failure case; and (c) a successful case.

Figure 2

An example of the processed image, and how the 4 classes are defined by the direction the red arrow points to.

Figure 3

Network architecture with feature maps of sizes {80, 40, 20, 10, 5}, and an example of the inference process. The image is labeled as “Left” and predicted as “Left”.

Figure 4

(a) The 2–D features where each point represents a type of product; (b) From the perspectives of both WCSS and silhouette, it is indicated that k is equal to 5; and (c) The clustering result when k is equal to 5.

Figure 5

The split result of our proposal. The dotted lines in the right two plots show the average silhouette values for the corresponding left two plots.

Figure 6

From top to bottom are the split results of the training data being biased towards (a) cluster 1; (b) cluster 3; (c) cluster 4; and (d) cluster 5, while the case of cluster 2 is omitted due to its similarity to cluster 1, as discussed in Section 3.3. The dotted lines in the right two plots show the average silhouette values for the corresponding left two plots. The worst case is shown in (b), where there is no intersection of clusters and the silhouette distributions are far apart.

Figure 7

Grad-CAMs for 4 classes (a) “Down”, (b) “Left”, (c) “Right”, and (d) “Up”. For each subplot, the input images are in the first row, the second row shows the results of our proposal, and the subsequent rows show the results of cases (a–d).

Figure 8

Design block diagram.

Figure 9

Experimental setup of our handheld device for real-time inference and display.