Towards the sustainable development of smart cities through mass video surveillance: A response to the COVID-19 pandemic

Abstract

Sustainable smart city initiatives around the world have recently had great impact on the lives of citizens and brought significant changes to society. More precisely, data-driven smart applications that efficiently manage sparse resources are offering a futuristic vision of smart, efficient, and secure city operations. However, the ongoing COVID-19 pandemic has revealed the limitations of existing smart city deployment; hence; the development of systems and architectures capable of providing fast and effective mechanisms to limit further spread of the virus has become paramount. An active surveillance system capable of monitoring and enforcing social distancing between people can effectively slow the spread of this deadly virus. In this paper, we propose a data-driven deep learning-based framework for the sustainable development of a smart city, offering a timely response to combat the COVID-19 pandemic through mass video surveillance. To implementing social distancing monitoring, we used three deep learning-based real-time object detection models for the detection of people in videos captured with a monocular camera. We validated the performance of our system using a real-world video surveillance dataset for effective deployment.

Keywords: COVID-19 pandemic; Sustainable cities; deep learning; object detection; social distancing; video surveillance.

Fig. 1

Impact of social distancing on COVID-19 outbreak [20].

Fig. 2

Real-time monitoring of social distancing in a sustainable smart city scenario.

Fig. 3

Architectural representation of Faster R-CNN.

Fig. 4

Architectural representation of SSD.

Fig. 5

Architecture summary of YOLO.

Fig. 6

Illustration of the proposed system. Real-time video data from an IP surveillance camera is directly fed into the system for social distancing monitoring. An audio-visual non-intrusive dismissible alert is generated for any violation.

Fig. 7

Perspective transformation representation.

Fig. 8

Algorithmic flow of the proposed system. OpenCV’s perspective transform routine is used for bird’s eye view transformation.

Fig. 9

Illustrating intersection over union (IoU).

Fig. 10

Precision and recall curves based on all-point and 11-point interpolation for pedestrian detection using various object detection models at IoU = 0.5: (a) Faster R-CNN, (b) YOLO, and (c) SSD.

Fig. 11

(a) Precision and recall curves for pedestrian detection using the YOLO object detection model over different IoU thresholds 0.50:0.05:0.95 [AP@[0.5,0.95].

Fig. 12

Illustrating pedestrian detection using (a) YOLO (b) Faster R-CNN and social distancing monitoring (left: surveillance footage with bounding boxes, right: top-down transformation (bird’s eye view) showing violations in red).