Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 May 23;23(11):4992.
doi: 10.3390/s23114992.

Model-Predictive Control for Omnidirectional Mobile Robots in Logistic Environments Based on Object Detection Using CNNs

Stefan-Daniel Achirei  1 , Razvan Mocanu  2 , Alexandru-Tudor Popovici  1 , Constantin-Catalin Dosoftei  2
Affiliations

Model-Predictive Control for Omnidirectional Mobile Robots in Logistic Environments Based on Object Detection Using CNNs

Stefan-Daniel Achirei et al. Sensors (Basel). .

Abstract

Object detection is an essential component of autonomous mobile robotic systems, enabling robots to understand and interact with the environment. Object detection and recognition have made significant progress using convolutional neural networks (CNNs). Widely used in autonomous mobile robot applications, CNNs can quickly identify complicated image patterns, such as objects in a logistic environment. Integration of environment perception algorithms and motion control algorithms is a topic subjected to significant research. On the one hand, this paper presents an object detector to better understand the robot environment and the newly acquired dataset. The model was optimized to run on the mobile platform already on the robot. On the other hand, the paper introduces a model-based predictive controller to guide an omnidirectional robot to a particular position in a logistic environment based on an object map obtained from a custom-trained CNN detector and LIDAR data. Object detection contributes to a safe, optimal, and efficient path for the omnidirectional mobile robot. In a practical scenario, we deploy a custom-trained and optimized CNN model to detect specific objects in the warehouse environment. Then we evaluate, through simulation, a predictive control approach based on the detected objects using CNNs. Results are obtained in object detection using a custom-trained CNN with an in-house acquired data set on a mobile platform and in the optimal control for the omnidirectional mobile robot.

Keywords: computer vision; convolutional neural networks; depth sensing; discretized-time model; navigation; object detection; omnidirectional mobile robots; predictive control algorithm.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Depth information for ZED 1 (top left), ZED 2i (top right), ZED mini (bottom left), and Intel RealSense D435i (bottom right).
Figure 2
Figure 2
Acquired frame (column 1) and augmentation results (columns 2 and 3).
Figure 3
Figure 3
Object detection and distance estimation in meters (top) and 3D point cloud mapping (bottom).
Figure 4
Figure 4
Bird’s eye view mapping of detected objects.
Figure 5
Figure 5
Illustrative block diagram of the control structure.
Figure 6
Figure 6
Coordinates system for control algorithm illustrating the used notations.
Figure 7
Figure 7
Detected objects with SSD architectures (1st row) and with YOLOv5 architecture (2nd row).
Figure 8
Figure 8
Simulation results of the model−predictive controller with LiDAR data and simulation of camera detection (test case I).
Figure 9
Figure 9
Simulation results of the model−predictive controller with LiDAR data and simulation of camera detection (test case II).
Figure 10
Figure 10
Simulation results of the model−predictive controller with LiDAR data and simulation of camera detection (test case III).
See this image and copyright information in PMC

References

    1. Dosoftei C.C., Popovici A.T., Sacaleanu P.R., Gherghel P.M., Budaciu C. Hardware in the Loop Topology for an Omnidirectional Mobile Robot Using Matlab in a Robot Operating System Environment. Symmetry. 2021;13:969. doi: 10.3390/sym13060969. - DOI
    1. Rubio F., Valero F., Llopis-Albert C. A review of mobile robots: Concepts, methods, theoretical framework, and applications. Int. J. Adv. Robot. Syst. 2019;16:1729881419839596. doi: 10.1177/1729881419839596. - DOI
    1. Patle B., Babu L G., Pandey A., Parhi D., Jagadeesh A. A review: On path planning strategies for navigation of mobile robot. Def. Technol. 2019;15:582–606. doi: 10.1016/j.dt.2019.04.011. - DOI
    1. Ngwenya T., Ayomoh M., Yadavalli S. Virtual Obstacles for Sensors Incapacitation in Robot Navigation: A Systematic Review of 2D Path Planning. Sensors. 2022;22:6943. doi: 10.3390/s22186943. - DOI - PMC - PubMed
    1. Wang Z., Tian G. Hybrid offline and online task planning for service robot using object-level semantic map and probabilistic inference. Inf. Sci. 2022;593:78–98. doi: 10.1016/j.ins.2022.01.058. - DOI

LinkOut - more resources