Synergistic Integration of Skeletal Kinematic Features for Vision-Based Fall Detection

Abstract

According to the World Health Organisation, falling is a major health problem with potentially fatal implications. Each year, thousands of people die as a result of falls, with seniors making up 80% of these fatalities. The automatic detection of falls may reduce the severity of the consequences. Our study focuses on developing a vision-based fall detection system. Our work proposes a new feature descriptor that results in a new fall detection framework. The body geometry of the subject is analyzed and patterns that help to distinguish falls from non-fall activities are identified in our proposed method. An AlphaPose network is employed to identify 17 keypoints on the human skeleton. Thirteen keypoints are used in our study, and we compute two additional keypoints. These 15 keypoints are divided into five segments, each of which consists of a group of three non-collinear points. These five segments represent the left hand, right hand, left leg, right leg and craniocaudal section. A novel feature descriptor is generated by extracting the distances from the segmented parts, angles within the segmented parts and the angle of inclination for every segmented part. As a result, we may extract three features from each segment, giving us 15 features per frame that preserve spatial information. To capture temporal dynamics, the extracted spatial features are arranged in the temporal sequence. As a result, the feature descriptor in the proposed approach preserves the spatio-temporal dynamics. Thus, a feature descriptor of size [m×15] is formed where m is the number of frames. To recognize fall patterns, machine learning approaches such as decision trees, random forests, and gradient boost are applied to the feature descriptor. Our system was evaluated on the UPfall dataset, which is a benchmark dataset. It has shown very good performance compared to the state-of-the-art approaches.

Keywords: ambient intelligence; assistive technology; fall detection; fall prevention; real-time monitoring; risk assessment; signal processing; video analysis; vision-based human activity recognition.

Figure 1

Major consequences of falls.

Figure 2

A general fall detection system.

Figure 3

AlphaPose network.

Figure 4

Keypoints considered.

Figure 5

Distance calculation on all segments.

Figure 6

Angle of inclination for all segments.

Figure 7

Angle between the non-collinear points of every segment.

Figure 8

Workflow of the proposed fall detection system.

Figure 9

General form of a confusion matrix.

Figure 10

Confusion matrices of the algorithms in the proposed work. (a) depicts the confusion matrix of the Decision tree algorithm, (b) depicts the confusion matrix of the Random Forest algorithm and (c) depicts the confusion matrix of the Gradient boost algorithm.

Figure 11

AUC-ROC Curves of the algorithms in the proposed work. (a) Decision tree algorithm. (b) Random forest algorithm. (c) Gradient boost algorithm.

Figure 12

Accuracy of decision tree at different depth.

Figure 13

Performance of Gradient Boost algorithm at different learning rates.

Figure 14

Results of the falling activity of the proposed system. (a–d) show the sequence of falling backwards, (e–h) show the sequence of falling forwards and (i–l) show the sequence of a falling sideways.

Figure 15

Results of the daily living activities of the proposed system. (a,b) show images of laying activity, (c,d) show images of sitting activity and (e,f) show images of picking up an object.