Multimodal wireless sensor network-based ambient assisted living in real homes with multiple residents

Abstract

Human activity recognition and behavior monitoring in a home setting using wireless sensor networks (WSNs) provide a great potential for ambient assisted living (AAL) applications, ranging from health and wellbeing monitoring to resource consumption monitoring. However, due to the limitations of the sensor devices, challenges in wireless communication and the challenges in processing large amounts of sensor data in order to recognize complex human activities, WSN-based AAL systems are not effectively integrated in the home environment. Additionally, given the variety of sensor types and activities, selecting the most suitable set of sensors in the deployment is an important task. In order to investigate and propose solutions to such challenges, we introduce a WSN-based multimodal AAL system compatible for homes with multiple residents. Particularly, we focus on the details of the system architecture, including the challenges of sensor selection, deployment, networking and data collection and provide guidelines for the design and deployment of an effective AAL system. We also present the details of the field study we conducted, using the systems deployed in two different real home environments with multiple residents. With these systems, we are able to collect ambient sensor data from multiple homes. This data can be used to assess the wellbeing of the residents and identify deviations from everyday routines, which may be indicators of health problems. Finally, in order to elaborate on the possible applications of the proposed AAL system and to exemplify directions for processing the collected data, we provide the results of several human activity inference experiments, along with examples on how such results could be interpreted. We believe that the experiences shared in this work will contribute towards accelerating the acceptance of WSN-based AAL systems in the home setting.

Figure 1.

Layout and sensor deployment of House A.

Figure 2.

Layout and sensor deployment of House B.

Figure 3.

Example deployments of ambient sensors considering the designated criteria. (a) House A; (b) House B.

Figure 4.

The interface for labeling the activities performed by the residents.

Figure 5.

Example sensor readings matched with true activities (24 h).

Figure 6.

System information flow.

Figure 7.

Activity and social interaction behavior patterns. (a) Sleeping and napping duration distributions of residents; (b) social interaction ratio of time spent together at home.

Figure 8.

Starting times of breakfast, lunch and dinner of a resident through 30 days.

Figure 9.

Confusion matrices for classification results of House A. (a) k-nearest neighbors (kNN); (b) decision tree (DT); (c) hidden Markov model (HMM); (d) multi-layer perceptron (MLP); (e) time-delay neural network (TDNN).

Figure 10.

Confusion matrices for classification results of House B. (a) kNN; (b) DT; (c) HMM; (d) MLP; (e) TDNN.

Figure 11.

Activity classification results in terms of standard metrics. (a) House A; (b) House B.