Social distance monitoring framework using deep learning architecture to control infection transmission of COVID-19 pandemic

Abstract

The recent outbreak of the COVID-19 affected millions of people worldwide, yet the rate of infected people is increasing. In order to cope with the global pandemic situation and prevent the spread of the virus, various unprecedented precaution measures are adopted by different countries. One of the crucial practices to prevent the spread of viral infection is social distancing. This paper intends to present a social distance framework based on deep learning architecture as a precautionary step that helps to maintain, monitor, manage, and reduce the physical interaction between individuals in a real-time top view environment. We used Faster-RCNN for human detection in the images. As the human's appearance significantly varies in a top perspective; therefore, the architecture is trained on the top view human data set. Moreover, taking advantage of transfer learning, a new trained layer is fused with a pre-trained architecture. After detection, the pair-wise distance between peoples is estimated in an image using Euclidean distance. The detected bounding box's information is utilized to measure the central point of an individual detected bounding box. A violation threshold is defined that uses distance to pixel information and determines whether two people violate social distance or not. Experiments are conducted using various test images; results demonstrate that the framework effectively monitors the social distance between peoples. The transfer learning technique enhances the overall performance of the framework by achieving an accuracy of 96% with a False Positive Rate of 0.6%.

Keywords: COVID-19; Deep learning; Faster-RCNN; Internet of things; Person detection; Social distancing; Top view; Transfer learning.

Fig. 1

Latest number of worldwide COVID-19 confirmed cases (R. O. Q. W. R. O. O. G. U. K, 2021).

Fig. 2

Graphical illustration of social distance impact: blue curve describes the possible confirmed cases (sick people). It is noticed that with proper social distance management, the peak point of the pattern is at a lower point. The distribution of the peak is very high when no social distance measures are adapted (adapted from (whiteboxanalytics, 2020)).

Fig. 3

Different social distance monitoring methods used in Literature. (a) (Yang D. & Renganathan V., 2020) implemented Faster-RCNN for monitoring social distance (b)-(d) (Punn N. S., 2020), (Ramadass et al., 2020), utilized YOLOv3 with Deepsort tracking algorithm for social distance monitoring.

Fig. 4

Overview of framework used for social distance monitoring using top view human data set. The overall system is divided into two modules i.e., Human detection and social distance monitoring.

Fig. 5

The overall flow chart of the developed framework utilized for monitoring of social distance with top view human data set.

Fig. 6

Human detection module comprises of Faster-RCNN (Ren S. & Girshick R., 2021) with ResNet-(He K. & Ren S., 2021).

Fig. 7

(a) Detected human bounding boxes via detection module, (b) shows the calculated centriod/central point of the bounding box, and (c), illustrates the estimated distance of every detected bounding box. In the sample image, the distance between every pairwise detected bounding box is indicated with white lines; the green circles show the central point.

Fig. 8

Results of pre-trained detection utilized for monitoring of social distance from the top view. In sample images, the peoples who maintain social distancing are detected in green rectangles. The people who violate and do not maintain social distance are presented in red rectangles. The manually yellow crosses points not detected results.

Fig. 9

Training accuracy of detection architecture after training on top view data set.

Fig. 10

Training loss of detection architecture after training on top view data set.

Fig. 11

Results of social distance monitoring after training and applying transfer learning. It can be observed that the performance of the detection architecture is enhanced. In sample images, the people who maintain social distance are detected with green bounding boxes, whereas those who do not maintain the social distance threshold are presented in red rectangles.

Fig. 12

Accuracy, recall, F1-score and precision results.

Fig. 13

True Positive rate of trained an pre-trained architecture.

Fig. 14

False Positive rate of trained an pre-trained architecture.