Human Fall Detection Based on Body Posture Spatio-Temporal Evolution

Abstract

Abnormal falls in public places have significant safety hazards and can easily lead to serious consequences, such as trampling by people. Vision-driven fall event detection has the huge advantage of being non-invasive. However, in actual scenes, the fall behavior is rich in diversity, resulting in strong instability in detection. Based on the study of the stability of human body dynamics, the article proposes a new model of human posture representation of fall behavior, called the "five-point inverted pendulum model", and uses an improved two-branch multi-stage convolutional neural network (M-CNN) to extract and construct the inverted pendulum structure of human posture in real-world complex scenes. Furthermore, we consider the continuity of the fall event in time series, use multimedia analytics to observe the time series changes of human inverted pendulum structure, and construct a spatio-temporal evolution map of human posture movement. Finally, based on the integrated results of computer vision and multimedia analytics, we reveal the visual characteristics of the spatio-temporal evolution of human posture under the potentially unstable state, and explore two key features of human fall behavior: motion rotational energy and generalized force of motion. The experimental results in actual scenes show that the method has strong robustness, wide universality, and high detection accuracy.

Keywords: computer vision; fall behavior detection; five-point inverted pendulum model; human posture spatio-temporal map; motion instability; rotational energy.

Figure 1

An overview of the proposed method for abnormal fall behavior detection.

Figure 2

Five-point inverted pendulum model. The left model indicates COCO model [2] that includes 18 key parts of orange color and the yellow dotted line for human skeleton and five-point inverted pendulum model that includes five purple stars and the red real line for human skeleton. In the test, different moving objects (or different persons) indicated by a five-point inverted pendulum model are shown with different colors.

Figure 3

The standard spatio-temporal evolution map based on five-point inverted pendulum model. The left shows spatio-temporal evolution map describes the human body motion at the complete time sequence.

Figure 4

A simple particle system composed of two particles of the end point $E{P}_1$ and end point $E{P}_2$.

Figure 5

The standard spatio-temporal evolution maps for eight directional falling and walking samples.

Figure 6

The two-connecting rod system based on five-point inverted pendulum mode. Here, $C_{(·)}$ is the counter-acting force and $G_{(·)}$ is the gravity.

Figure 7

The extraction results of standard spatio-temporal evolution map in (5–10) scenarios. The left are input images from each the scenario; the middle are spatio-temporal evolution maps which describe the human body motion at the time sequence; the right are the standard spatio-temporal evolution maps (if the rotational energy is less than the threshold value, the standard spatio-temporal evolution map does not display for improving the efficiency of algorithms)

Figure 8

The curves of Receiver Operating Characteristic (ROC) and Precision–Recall (P–R) for three algorithms.

Figure 9

The radar charts of the characteristics in eight directional falling and walking samples.

Figure 10

The results of Radviz data analysis.