Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2021 Feb 24;21(5):1571.
doi: 10.3390/s21051571.

Human-Robot Perception in Industrial Environments: A Survey

Andrea Bonci  1 , Pangcheng David Cen Cheng  2 , Marina Indri  2 , Giacomo Nabissi  1 , Fiorella Sibona  2
Affiliations
Review

Human-Robot Perception in Industrial Environments: A Survey

Andrea Bonci et al. Sensors (Basel). .

Abstract

Perception capability assumes significant importance for human-robot interaction. The forthcoming industrial environments will require a high level of automation to be flexible and adaptive enough to comply with the increasingly faster and low-cost market demands. Autonomous and collaborative robots able to adapt to varying and dynamic conditions of the environment, including the presence of human beings, will have an ever-greater role in this context. However, if the robot is not aware of the human position and intention, a shared workspace between robots and humans may decrease productivity and lead to human safety issues. This paper presents a survey on sensory equipment useful for human detection and action recognition in industrial environments. An overview of different sensors and perception techniques is presented. Various types of robotic systems commonly used in industry, such as fixed-base manipulators, collaborative robots, mobile robots and mobile manipulators, are considered, analyzing the most useful sensors and methods to perceive and react to the presence of human operators in industrial cooperative and collaborative applications. The paper also introduces two proofs of concept, developed by the authors for future collaborative robotic applications that benefit from enhanced capabilities of human perception and interaction. The first one concerns fixed-base collaborative robots, and proposes a solution for human safety in tasks requiring human collision avoidance or moving obstacles detection. The second one proposes a collaborative behavior implementable upon autonomous mobile robots, pursuing assigned tasks within an industrial space shared with human operators.

Keywords: 3D sensors; collision avoidance; collision detection; human action recognition; human-robot collaboration; human-robot perception; machine vision; robot guidance.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Service robots for professional use. Top 3 applications unit sales 2018 and 2019, potential development 2020–2023 (thousands of units) [30].
Figure 2
Figure 2
Relevant sensors for HRP within the industrial context. Vision sensors are highlighted in blue, safety laser scanners rays in red, and wearable sensors in orange.
Figure 3
Figure 3
Most relevant sensors for HRP which can be found on the robot.
Figure 4
Figure 4
Most relevant sensors for HRP which can be found on the human operator.
Figure 5
Figure 5
Architecture setup for obstacle detection at different working range. Data stream, processing and communication between sensor node and robot controller are showed.
Figure 6
Figure 6
Block diagram representing the logical workflow and algorithms’ interaction of the POC.
Figure 7
Figure 7
Simulated (left) and real (right) KUKA collaborative robot used on the proposed application.
Figure 8
Figure 8
Schematic view of the calibration process with representation of sensor workspace and robot workspace. The axes measurement units are assumed in m.
Figure 9
Figure 9
(Left) Mesh primitives (green colored) representing bounding volumes of collision envelope overlapped to the robot model. (Right) Collision envelope meshes attached to the robot structure using DH convention and collision box. The axes measurement units are assumed in m.
Figure 10
Figure 10
RGB-D data processing and occupancy map reconstruction. (a) Depth space image acquisition of Mir200 AGV. (b) Human detection in long-range with YOLO CNN based on RGB image. (c) Occupancy map created with Octotree structure, the axes measurement units are assumed as 1 unit = 10−1 m. (d) Occupancy map with inflation radius of 0.2 m, the axes measurement units are assumed as 1 unit = 10−1 m.
Figure 11
Figure 11
Workflow for activating the collaborative mode of the Sen3bot. The grey area represents the intersection of the vision and laser sensors FOV. (a) A human operator within the monitored area of type 2 needs assistance from one of the Sen3Bots. (b) Given the led blue color, the operator identifies the Sen3Bot ready for collaboration. (c) The human operator stops at a distance Df allowing the robot to scan its front QR code. (d) The human operator turns around allowing the robot to scan its back QR code. (e) The green indicator light indicates that the mobile robot entered the collaborative mode.
Figure 12
Figure 12
Modes switching schema for a Sen3Bot monitoring an area of type 2, enabled to wait for collaborative task triggering, i.e., with wait4col ==1.
See this image and copyright information in PMC

References

    1. Wu X., Goepp V., Siadat A. Cyber Physical Production Systems: A Review of Design and Implementation Approaches; Proceedings of the 2019 IEEE International Conference on Industrial Engineering and Engineering Management (IEEM); Macao, China. 15–18 December 2019; pp. 1588–1592.
    1. Bonci A., Pirani M., Longhi S. An embedded database technology perspective in cyber-physical production systems. Procedia Manuf. 2017;11:830–837. doi: 10.1016/j.promfg.2017.07.185. - DOI
    1. Bonci A., Pirani M., Dragoni A.F., Cucchiarelli A., Longhi S. The relational model: In search for lean and mean CPS technology; Proceedings of the 2017 IEEE 15th International Conference on Industrial Informatics (INDIN); Emden, Germany. 24–26 July 2017; pp. 127–132.
    1. Bonci A., Pirani M., Cucchiarelli A., Carbonari A., Naticchia B., Longhi S. A review of recursive holarchies for viable systems in CPSs; Proceedings of the 2018 IEEE 16th International Conference on Industrial Informatics (INDIN); Porto, Portugal. 18–20 July 2018; pp. 37–42.
    1. Yildiz Y. Cyberphysical Human Systems: An Introduction to the Special Issue. IEEE Control Syst. Mag. 2020;40:26–28.