Congested Crowd Counting via Adaptive Multi-Scale Context Learning

Abstract

In this paper, we propose a novel congested crowd counting network for crowd density estimation, i.e., the Adaptive Multi-scale Context Aggregation Network (MSCANet). MSCANet efficiently leverages the spatial context information to accomplish crowd density estimation in a complicated crowd scene. To achieve this, a multi-scale context learning block, called the Multi-scale Context Aggregation module (MSCA), is proposed to first extract different scale information and then adaptively aggregate it to capture the full scale of the crowd. Employing multiple MSCAs in a cascaded manner, the MSCANet can deeply utilize the spatial context information and modulate preliminary features into more distinguishing and scale-sensitive features, which are finally applied to a 1 × 1 convolution operation to obtain the crowd density results. Extensive experiments on three challenging crowd counting benchmarks showed that our model yielded compelling performance against the other state-of-the-art methods. To thoroughly prove the generality of MSCANet, we extend our method to two relevant tasks: crowd localization and remote sensing object counting. The extension experiment results also confirmed the effectiveness of MSCANet.

Keywords: crowd counting; crowd density estimation; crowd localization; multi-scale context learning; remote sensing object counting.

Figure 1

Representative examples in the UCF-QNRF dataset [17]. From left to right: input images, ground-truth, results of CSRNet [8], and the results of MSCANet. Compared to CSRNet, MSCANet can effectively handle the ambiguity of appearance between crowd and background objects.

Figure 2

Detailedillustration of our Adaptive Multi-scale Context Aggregation Network for crowd counting.

Figure 3

Different structures of multi-scale context modules. (a) Multi-scale context aggregation module (MSCA) w/o channel attention (CA); (b) cascade context pyramid module (CCPM); (c) scale pyramid module (SPM); and (d) scale-aware context module (SACM).

Figure 4

Visualizations of MSCANet for crowd localization on the UCF-QNRF dataset. Red points denote the ground-truth, and green points denote the estimated location results of MSCANet.

Figure 5

Visualization results of MSCANet for remote sensing object counting on RSOC dataset.

Figure 6

Impacts of different pyramid scale settings on UCF-QNRF. From left to right: input image, ground truth, result of PS = {1}, result of PS = {1,2}, result of PS = {1,2,3}, and result of PS = {1,2,3,4}.

Figure 7

Impacts of CA on UCF-QNRF. From left to right: input image, ground-truth, result of MSCA w/o CA, and result of MSCA.

Figure 8

Visual comparision of different multi-scale context modules on UCF-QNRF. From left to right: input images, ground-truth, results of our method, results of CCPM, results of SPM, and results of SACM.