Multiscale regional calibration network for crowd counting

Abstract

Crowd counting aims to estimate the number, density, and distribution of crowds in an image. While CNN-based crowd counting methods have been effective, head-scale variation and complex background remain two major challenges for crowd counting. Therefore, we propose a multiscale region calibration network called MRCNet to effectively address these challenges. To address the former challenge, we design a multiscale aware module that utilizes multi-branch dilated convolutional parallelism to obtain multiscale receptive fields and cope with drastic changes in head size. For the latter challenge, we design a regional calibration module that calibrates the attention weights of each region after obtaining the attention map to effectively handle challenges in complex contexts. Additionally, we improve the loss function by combining L2 loss and binary cross-entropy loss to help MRCNet achieve excellent results. Extensive experiments were conducted on three mainstream datasets to demonstrate the robustness and competitiveness of our approach.

Keywords: Crowd counting; Feature aggregation; Multiscale; Regional calibration.

Fig. 1

Major challenges facing current crowd counting tasks. (a) The problem of head-size variation. (b) The problem of complex background.

Fig. 2

The overall structure of MRCNet. Firstly, the images are fed into the Feature Extraction Module (FEM), which utilizes the first 13 layers of VGG-16 for initial feature extraction. The FEM outputs three levels of features with different network depths, which are then passed through the Multiscale Aware Module (MAM) and Regional Calibration Module (RCM) for feature enhancement. This process generates three levels of features, denoted as , , and . Finally, the three levels of features are combined in the Feature Aggregation Module (FAM) to produce the predicted density map. An improved loss function is used to train the network.

Fig. 3

Visualization results on different datasets, where the three columns of pictures from left to right are the true image, the ground-truth density map, and the predicted density map.

Fig. 4

The visualization results of different combinations of MAM and RCM are shown as follows: (a) After adding the MAM, the network can adapt to drastic changes in head-size. (b) When the scene is complex, the network has large errors, such as counting leaves as heads. By adding RCM, the network can enhance its counting accuracy and effectively suppress background interference.