Automated Cognitive Health Assessment Using Partially Complete Time Series Sensor Data

Abstract

Background: Behavior and health are inextricably linked. As a result, continuous wearable sensor data offer the potential to predict clinical measures. However, interruptions in the data collection occur, which create a need for strategic data imputation.

Objective: The objective of this work is to adapt a data generation algorithm to impute multivariate time series data. This will allow us to create digital behavior markers that can predict clinical health measures.

Methods: We created a bidirectional time series generative adversarial network to impute missing sensor readings. Values are imputed based on relationships between multiple fields and multiple points in time, for single time points or larger time gaps. From the complete data, digital behavior markers are extracted and are mapped to predicted clinical measures.

Results: We validate our approach using continuous smartwatch data for n = 14 participants. When reconstructing omitted data, we observe an average normalized mean absolute error of 0.0197. We then create machine learning models to predict clinical measures from the reconstructed, complete data with correlations ranging from r = 0.1230 to r = 0.7623. This work indicates that wearable sensor data collected in the wild can be used to offer insights on a person's health in natural settings.

Figure 1.

The process of assessing health from smartwatch data. Data are continuously collected while a participant wears a smartwatch and performs their normal routine. Data are securely stored in a relational database and a processed by imputing missing values (Mink), labeling readings with associated activity labels (activity recognition), and extracting a set of digital behavior markers. A machine learning then maps the behavior profile onto predicted clinical measures.

Figure 2.

The Mink time series data imputation architecture. The system processes time series X containing a mixture of observed and missing values and outputs a complete time series $\widehat{X}$ with no missing values. To generate realistic data, Mink combines an autoencoder (with embedding function e and recovery function r) and a generative adversarial network (with generator g and discriminator d).