Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Review
. 2019 Dec 19;20(1):43.
doi: 10.3390/s20010043.

Convolutional-Neural Network-Based Image Crowd Counting: Review, Categorization, Analysis, and Performance Evaluation

Naveed Ilyas  1 , Ahsan Shahzad  2 , Kiseon Kim  1
Affiliations
Review

Convolutional-Neural Network-Based Image Crowd Counting: Review, Categorization, Analysis, and Performance Evaluation

Naveed Ilyas et al. Sensors (Basel). .

Abstract

Traditional handcrafted crowd-counting techniques in an image are currently transformed via machine-learning and artificial-intelligence techniques into intelligent crowd-counting techniques. This paradigm shift offers many advanced features in terms of adaptive monitoring and the control of dynamic crowd gatherings. Adaptive monitoring, identification/recognition, and the management of diverse crowd gatherings can improve many crowd-management-related tasks in terms of efficiency, capacity, reliability, and safety. Despite many challenges, such as occlusion, clutter, and irregular object distribution and nonuniform object scale, convolutional neural networks are a promising technology for intelligent image crowd counting and analysis. In this article, we review, categorize, analyze (limitations and distinctive features), and provide a detailed performance evaluation of the latest convolutional-neural-network-based crowd-counting techniques. We also highlight the potential applications of convolutional-neural-network-based crowd-counting techniques. Finally, we conclude this article by presenting our key observations, providing strong foundation for future research directions while designing convolutional-neural-network-based crowd-counting techniques. Further, the article discusses new advancements toward understanding crowd counting in smart cities using the Internet of Things (IoT).

Keywords: crowd analysis; deep learning; smart cities.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Categorization of crowd-counting techniques.
Figure 2
Figure 2
Unique challenges of convolutional-neural-network (CNN) crowd counting (CC) techniques in an image.
Figure 3
Figure 3
General form of CNN-CC algorithm. Crowd-counting mechanism starts from object annotation in an image to density estimation; object counting is depicted. General framework of crowd counting (top), and CNN working is expanded (bottom).
Figure 4
Figure 4
Categorization of CNN-CC techniques.
Figure 5
Figure 5
Architectures of different subcategories: (a) basic-CNN-CC, (b) context-aware CNN-CC techniques (context-CNN-CC), (c) patch-based-CNN-CC, (d) scale-aware CNN-CC techniques (scale-CNN-CC), (e) multitask-CNN-CC, (f) whole-image-CNN-CC, (g) aerial-view-CNN-CC, and (h) perspective-CNN-CC.
Figure 6
Figure 6
Applications of crowd analysis in different fields.
Figure 7
Figure 7
Normalized Mean Absolute Error (nMAE) of network-CNN-CC algorithms tested on different datasets: (a) basic-CNN-CC, (b) context-CNN-CC, (c) scale-CNN-CC, and (d) multitask-CNN-CC.
Figure 8
Figure 8
nMAE of CNN-CC algorithms tested on different datasets: (a) perspective-CNN-CC, (b) patch-based-CNN-CC, and (c) whole-image-CNN-CC.
See this image and copyright information in PMC

References

    1. Wang H., Roa A.C., Basavanhally A.N., Gilmore H.L., Shih N., Feldman M., Tomaszewski J., Gonzalez F., Madabhushi A. Mitosis detection in breast cancer pathology images by combining handcrafted and convolutional neural network features. J. Med. Imaging. 2014;1:034003. doi: 10.1117/1.JMI.1.3.034003. - DOI - PMC - PubMed
    1. Wang H., Cruz-Roa A., Basavanhally A., Gilmore H., Shih N., Feldman M., Tomaszewski J., Gonzalez F., Madabhushi A. Medical Imaging 2014: Digital Pathology. Volume 9041. International Society for Optics and Photonics; San Diego, CA, USA: 2014. Cascaded ensemble of convolutional neural networks and handcrafted features for mitosis detection; p. 90410B.
    1. Dollar P., Wojek C., Schiele B., Perona P. Pedestrian detection: An evaluation of the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 2012;34:743–761. doi: 10.1109/TPAMI.2011.155. - DOI - PubMed
    1. Chen K., Loy C.C., Gong S., Xiang T. Feature mining for localised crowd counting; Proceedings of the BMVC; Surrey, UK. 3–7 September 2012; p. 3.
    1. Fiaschi L., Köthe U., Nair R., Hamprecht F.A. Learning to count with regression forest and structured labels; Proceedings of the 2012 21st International Conference on Pattern Recognition (ICPR); Tsukuba Science City, Japan. 11–15 November 2012; pp. 2685–2688.

LinkOut - more resources