Pose Estimation of a Cobot Implemented on a Small AI-Powered Computing System and a Stereo Camera for Precision Evaluation

Abstract

The precision of robotic manipulators in the industrial or medical field is very important, especially when it comes to repetitive or exhaustive tasks. Geometric deformations are the most common in this field. For this reason, new robotic vision techniques have been proposed, including 3D methods that made it possible to determine the geometric distances between the parts of a robotic manipulator. The aim of this work is to measure the angular position of a robotic arm with six degrees of freedom. For this purpose, a stereo camera and a convolutional neural network algorithm are used to reduce the degradation of precision caused by geometric errors. This method is not intended to replace encoders, but to enhance accuracy by compensating for degradation through an intelligent visual measurement system. The camera is tested and the accuracy is about one millimeter. The implementation of this method leads to better results than traditional and simple neural network methods.

Keywords: 3D vision; convolutional neural network; precision degradation; robot arm; robotic vision; stereo camera.

Figure 1

A 3-DoF robot architecture and its principal components, this model is simplified to understand the main concepts [1].

Figure 2

Schematic of stereo vision configuration in three dimensions [16,32].

Figure 3

Triangulation scheme analysis for stereo vision a two dimensional perspective [16].

Figure 4

Matching windows in stereo vision. This represents the digital functionality of a commercial stereo camera [33].

Figure 5

Distance of 1 cm in pixels, and the subsequent pixel to measure the D435 camera.

Figure 6

Configuration of the materials, the rectangle represents a table of 56 × 150 cm, the triangle is the view of the stereo camera, the circle is the workspace of the robot.

Figure 7

6 DoF Cobot morphology. Only 4 DoF are labeled with a different color.

Figure 8

GUI used for better manipulation of the robot arm, for communication; RPC (Remote Procedure Call) protocol is implemented, RPC communication is based on TCP (Transfer Control Protocol) architecture.

Figure 9

Graphical displaying of 1 DoF position accuracy according to ISO 9283-1998 [34].

Figure 10

Workspace of the Cobot 280 Nano.

Figure 11

LENET-5 architecture [35].

Figure 12

AlexNet architecture [36].

Figure 13

Every three-dimensional position of TCP position (x, y, z), the fourth graph is the points that the Cobot follows.

Figure 14

Design of the 1-dimensional CNN used for predicting inverse kinematics, highlighting the input layer, convolutional layers, and output.

Figure 15

Training vs. validation loss function of the 1 dimensional CNN.

Figure 16

Nine samples of RGB pictures of the Cobot, the pixels are 480 × 640 pixels.

Figure 17

Nine samples of depth information for the Cobot, it has the same size of a color image, every pixel has a resolution of 16 bits.

Figure 18

Training vs. validation loss function of LENET-5.

Figure 19

Training vs. validation loss function of AlexNet.

Figure 20

Images of the robot with colored labels, using YOLO-V1.

Figure 21

Yolo-V1 training results for label identification. (a) Mean average precision (mAP) is used to measure the performance of computer vision models. (b) Box Loss refers to how well the model predicts the positions and sizes of bounding boxes around objects in an image. (c) Class Loss refers to the measure of how accurately the model is predicting the correct class or label of the objects it detects. (d) Object Loss measures how well the model recognizes the presence or absence of an object in a particular region of an image.

Figure 22

Results with a new set of images to evaluate the Yolo-V1 algorithm. The red dots are the recognized labels.

Figure 23

Depth information of the color images.

Figure 24

The image on the left is the depth information of the color images with the location of every label. The color box represents the workspace where labels are projected. The dots in the color box are the marked points detected by the YOLO-V1 algorithm that helps measure each DoF angular position, where blue dot is the first DoF label, green dot is the second DoF label, yellow dot is the third DoF label and finally, the red dot is the fourth DoF label.