Design and Analysis of a Lightweight Context Fusion CNN Scheme for Crowd Counting

Abstract

Crowd counting, which is widely used in disaster management, traffic monitoring, and other fields of urban security, is a challenging task that is attracting increasing interest from researchers. For better accuracy, most methods have attempted to handle the scale variation explicitly. which results in huge scale changes of the object size. However, earlier methods based on convolutional neural networks (CNN) have focused primarily on improving accuracy while ignoring the complexity of the model. This paper proposes a novel method based on a lightweight CNN-based network for estimating crowd counting and generating density maps under resource constraints. The network is composed of three components: a basic feature extractor (BFE), a stacked à trous convolution module (SACM), and a context fusion module (CFM). The BFE encodes basic feature information with reduced spatial resolution for further refining. Various pieces of contextual information are generated through a short pipeline in SACM. To generate a context fusion density map, CFM distills feature maps from the above components. The whole network is trained in an end-to-end fashion and uses a compression factor to restrict its size. Experiments on three highly-challenging datasets demonstrate that the proposed method delivers attractive performance.

Keywords: computer vision; convolutional neural networks; crowd counting; deep learning.

Figure 1

The architecture of the basic unit in our backbone. (a) is the layer organization in which a unit is stacked repeatedly and the shortcut connection aids information transfer, and (b) is the scheme of the unit, which is composed of a $1\times 1$ bottleneck layer and a $3\times 3$ ordinary convolutional layer. The compression factor ($\alpha$) and the filters of the Unit ($filters$) jointly determine the number of filters in each layer.

Figure 2

Overview of the proposed network for density estimation. The network extracts common feature maps with reduced spatial resolution using basic feature extractor (BFE). To ensure a diversity of contextual features, the stacked à trous convolution module (SACM) enlarges the gap between the receptive sizes of the feature maps. The context fusion module (CFM) distills different contextual information and fuses it to estimate the density map.

Figure 3

Analysis of the context fusion module (CFM). The outputs of CFM are visualized, and the red bounding boxes mark the same spatial region in different maps. In the feature maps, the color from red to blue reflects the response from strong to weak. (a) Input image, (b) estimation result, (c) first output of CFM, (d) second output of CFM, and (e) third output of CFM.

Figure 4

Density estimation results on the ShanghaiTech Part A [8]. (a) Input images, (b) ground truth density maps, and (c) estimation results.

Figure 5

Density estimation results on ShanghaiTech Part B [8]. (a) Input images, (b) ground truth density maps, and (c) estimation results.

Figure 6

Density estimation results on the UCF_CC_50 [3]. (a) Input images, (b) ground truth density maps, and (c) estimation results.

Figure 7

Density estimation results on the WorldExpo’10 dataset [14]. (a) Input images, (b) ground truth density maps, and (c) estimation results.