Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2019 May 2;19(9):2048.
doi: 10.3390/s19092048.

Edge-Computing Video Analytics for Real-Time Traffic Monitoring in a Smart City

Johan Barthélemy  1 , Nicolas Verstaevel  2 , Hugh Forehead  3 , Pascal Perez  4
Affiliations

Edge-Computing Video Analytics for Real-Time Traffic Monitoring in a Smart City

Johan Barthélemy et al. Sensors (Basel). .

Abstract

The increasing development of urban centers brings serious challenges for traffic management. In this paper, we introduce a smart visual sensor, developed for a pilot project taking place in the Australian city of Liverpool (NSW). The project's aim was to design and evaluate an edge-computing device using computer vision and deep neural networks to track in real-time multi-modal transportation while ensuring citizens' privacy. The performance of the sensor was evaluated on a town center dataset. We also introduce the interoperable Agnosticity framework designed to collect, store and access data from multiple sensors, with results from two real-world experiments.

Keywords: CCTV; IoT; edge-computing; smart city; traffic monitoring; video analytic.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest. The founding sponsors had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, and in the decision to publish the results.

Figures

Figure 1
Figure 1
A map of the town center. (left) Locations of the 20 visual sensors. It is expected that the live data produced from those location will help urban planners to update the city’s mobility plan. (right) Co-location of two CCTV cameras and an air quality (in purple) on a pole.
Figure 2
Figure 2
The smart visual sensor, outside and inside.
Figure 3
Figure 3
Simplified diagram of the sensor. The core of the sensor is the Jetson TX2 handling the video analytics, the USB and Ethernet connectivity and powering the LoPy 4 and the OLED screen. The Jetson TX2 is in turn powered by the PSU. The LoPy is used for LoRaWAN-based transmission and the screen to display basic information about the sensor’s status.
Figure 4
Figure 4
Activity flow chart of the sensor.
Figure 5
Figure 5
The 3U server version of the visual sensor hosting 15 NVIDIA Jetson TX2 units, each of these computing modules are able to process in real time one live video feed from a CCTV camera.
Figure 6
Figure 6
Architecture of YOLO V3 for object detection. A picture is passed through a fully convolutional neural network of 406 hidden layers in order to predict box coordinates and class probabilities at three different scales (large, medium, small). A non-maximum suppression algorithm is then applied to only retain the category and coordinates with the higher score.
Figure 7
Figure 7
The general architecture of the project. The Agnosticity software stack relies on well established open-source software. The data collection and access is ensured by the open-source implementation of the OM2M standard.
Figure 8
Figure 8
The web-based interactive dashboard used to represent the collected data from the different sensors deployed for the Liverpool project. The interface is responsive and can be used both on desktop and mobile browsers.
Figure 9
Figure 9
Kernel density estimation of the accuracy (left) and percentage deviation (right) computed across the 4500 frames of the Oxford dataset.
Figure 10
Figure 10
Evolution of the number of frame per seconds processed (FPS) by sensor (blue) and the number of detected objects (red) over time. It can be seen that the FPS are higher when the number of detection is lower. The drop in FPS is mainly due to the SORT algorithm as there are more objects to be tracked, a task not taking advantage of the CUDA cores available on the Jetson TX2.
Figure 11
Figure 11
Evolution of the accuracy (blue lines, ideal is 1.0) in accordance with the ground truth (orange lines) over time. Accuracy is better with small groups and decreases with large crowds.
Figure 12
Figure 12
Number of ground truth detection against the number of detection. We observe that the relationship is linear, meaning that the algorithm manages to capture trends.
Figure 13
Figure 13
A 15 min monitoring of the CPU, GPU, memory and disk usages, average temperature (top) and network utilization (bottom) during a real world deployment of the sensor. Over this period, 280 unique objects have been detected.
Figure 14
Figure 14
Plot of the number of people detected inside a building over one hour. The two peaks correspond to the start and end of the fire alarm event.
Figure 15
Figure 15
Trajectories followed by the individuals detected and tracked by the sensor. Each of the 631 lines represent one individual.
Figure 16
Figure 16
Heat map of the maximum number of individual detected in the field of view of the sensor.
Figure 17
Figure 17
The sensor is located at the center of the city, next to a pedestrian street (highlighted in red). The field of view of the camera is represented in blue.
Figure 18
Figure 18
Number of pedestrian, vehicle and bicycle detected by the sensor between 20 February 2019 and 27 February 2019. Each data point represents the number of objects of a specific type detected over the last minute.
Figure 19
Figure 19
Number of detection pedestrians (green), bicycles (red), and vehicles (yellow) detected hourly on 23 February 2019.
Figure 20
Figure 20
Plot of the pixel coordinates (X,Y) of the detected pedestrians (blue) or bicycles (orange) in the frame for the day 23 February 2019. Each dot represents the centroid of the bounding box corresponding to an item that has been detected at those coordinates.
Figure 21
Figure 21
Trajectories of pedestrians and bicycles within the frame during the day of 23 February 2019. Two flows of pedestrians are visible.
See this image and copyright information in PMC

References

    1. Nations U. World Population Prospects: The 2017 Revision, Key Findings and Advance Tables. Departement of Economic and Social Affaire; New York City, NY, USA: 2017.
    1. Bulkeley H., Betsill M. Rethinking sustainable cities: Multilevel governance and the’urban’politics of climate change. Environ. Politics. 2005;14:42–63. doi: 10.1080/0964401042000310178. - DOI
    1. Albino V., Berardi U., Dangelico R.M. Smart cities: Definitions, dimensions, performance, and initiatives. J. Urban Technol. 2015;22:3–21. doi: 10.1080/10630732.2014.942092. - DOI
    1. Bibri S.E., Krogstie J. Smart sustainable cities of the future: An extensive interdisciplinary literature review. Sustain. Cities Soc. 2017;31:183–212. doi: 10.1016/j.scs.2017.02.016. - DOI
    1. Bibri S.E., Krogstie J. On the social shaping dimensions of smart sustainable cities: A study in science, technology, and society. Sustain. Cities Soc. 2017;29:219–246. doi: 10.1016/j.scs.2016.11.004. - DOI

LinkOut - more resources