Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2013 Oct 29;13(11):14687-713.
doi: 10.3390/s131114687.

On the use of a low-cost thermal sensor to improve Kinect people detection in a mobile robot

Loreto Susperregi  1 , Basilio Sierra, Modesto Castrillón, Javier Lorenzo, Jose María Martínez-Otzeta, Elena Lazkano
Affiliations

On the use of a low-cost thermal sensor to improve Kinect people detection in a mobile robot

Loreto Susperregi et al. Sensors (Basel). .

Abstract

Detecting people is a key capability for robots that operate in populated environments. In this paper, we have adopted a hierarchical approach that combines classifiers created using supervised learning in order to identify whether a person is in the view-scope of the robot or not. Our approach makes use of vision, depth and thermal sensors mounted on top of a mobile platform. The set of sensors is set up combining the rich data source offered by a Kinect sensor, which provides vision and depth at low cost, and a thermopile array sensor. Experimental results carried out with a mobile platform in a manufacturing shop floor and in a science museum have shown that the false positive rate achieved using any single cue is drastically reduced. The performance of our algorithm improves other well-known approaches, such as C4 and histogram of oriented gradients (HOG).

PubMed Disclaimer

Figures

Figure 1.
Figure 1.
The robotic platform used: a Segway RMP200 provided with the Kinect and the thermal sensor.
Figure 2.
Figure 2.
Detection results using Kinect algorithms: IK4-TEKNIKER dataset.
Figure 3.
Figure 3.
First phase: learning classifiers from three transformed data. Computer vision transformations over the original images are performed to enrich the input database sources.
Figure 4.
Figure 4.
HTPAthermopile image sample and a miniature of its corresponding RGB image.
Figure 5.
Figure 5.
Image preprocessing and training database creation from a hand-labeled original dataset and transformed images.
Figure 6.
Figure 6.
Positive examples in the three data sources (intensity, depth, thermal) with people with different sizes and positions.
Figure 6.
Figure 6.
Positive examples in the three data sources (intensity, depth, thermal) with people with different sizes and positions.
Figure 7.
Figure 7.
Hierarchical classifier schemata.
Figure 8.
Figure 8.
Manufacturing plant at IK4-TEKNIKER.
Figure 9.
Figure 9.
Negative examples in the three data sources (intensity, depth, thermal), with different elements in the environment.
Figure 10.
Figure 10.
Images from the Eureka! Science Museum in the three data sources (intensity, depth, thermal), where different issues relevant to the problem are represented. From the left: many persons, people and objects with similar silhouettes and Sun incidence in corridors.
Figure 11.
Figure 11.
Respectively, a true positive, false positive, false negative and true negative example using our approach.
Figure 12.
Figure 12.
Respectively, a true positive, false positive, false negative and true negative example using the C4 approach.
Figure 13.
Figure 13.
Example of a hierarchical classifier.
See this image and copyright information in PMC

References

    1. Shotton J., Fitzgibbon A., Cook M., Sharp T., Finocchio M., Moore R., Kipma A., Blake A. Real-Time Human Pose Recognition in Parts from a Single Depth Image. Proceedings of the Computer Vision and Pattern Recognition, Colorado Springs; CO, USA. 20–25 June 2011.
    1. Kinect Sensor. [(accessed on 10 June 2013)]. Available online: http://en.wikipedia.org/wiki/Kinect.
    1. Khoshelham K., Elberink S.O. Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors. 2012;12:1437–1454. - PMC - PubMed
    1. Heimann Sensor. [(accessed on 20 March 2013).]. Available online: http://www.heimannsensor.com/index.php.
    1. Bellotto N., Hu H. A bank of unscented Kalman filters for multimodal human perception with mobile service robots. Int. J. Soc. Robot. 2010;2:121–136.

Publication types

MeSH terms