Improved Mental Acuity Forecasting with an Individualized Quantitative Sleep Model

Abstract

Sleep impairment significantly alters human brain structure and cognitive function, but available evidence suggests that adults in developed nations are sleeping less. A growing body of research has sought to use sleep to forecast cognitive performance by modeling the relationship between the two, but has generally focused on vigilance rather than other cognitive constructs affected by sleep, such as reaction time, executive function, and working memory. Previous modeling efforts have also utilized subjective, self-reported sleep durations and were restricted to laboratory environments. In the current effort, we addressed these limitations by employing wearable systems and mobile applications to gather objective sleep information, assess multi-construct cognitive performance, and model/predict changes to mental acuity. Thirty participants were recruited for participation in the study, which lasted 1 week. Using the Fitbit Charge HR and a mobile version of the automated neuropsychological assessment metric called CogGauge, we gathered a series of features and utilized the unified model of performance to predict mental acuity based on sleep records. Our results suggest that individuals poorly rate their sleep duration, supporting the need for objective sleep metrics to model circadian changes to mental acuity. Participant compliance in using the wearable throughout the week and responding to the CogGauge assessments was 80%. Specific biases were identified in temporal metrics across mobile devices and operating systems and were excluded from the mental acuity metric development. Individualized prediction of mental acuity consistently outperformed group modeling. This effort indicates the feasibility of creating an individualized, mobile assessment and prediction of mental acuity, compatible with the majority of current mobile devices.

Keywords: actigraphy; cognition; executive function; machine learning; mobile applications; sleep.

Figure 1

Boxplot of self-reported sleep duration compared to Fitbit-reported sleep duration. Survey response options included <2, 2–4, 4–5, 6–7, and 8+ h. Blue bars represent participants who selected the survey response that accurately described their average sleep duration, as verified by the wearable device. Red bars represent participants who selected an incorrect description of their average sleep duration on the survey, and gray bars represent users who selected one of two possible correct responses.

Figure 2

Compliance with wearing the Fitbit device and responding to the CogGauge assessments among all 30 participants over the week long course of the experiment. Six participants who did not wear the device, did not sync the device, or had a broken device were removed from model development.

Figure 3

Top panel—Logarithmic boxplot of psychomotor vigilance test (PVT) response times (RTs) by device type. iPhone devices (iP5s, iP6+) recorded lower RTs than Samsung Galaxy S5 or MotoX devices. p-Values indicated between devices per Kruskal–Wallis testing. Standard PVT lapse threshold of 500 ms shown by horizontal line. Bottom panel—Phone RT discretization. Histograms of RT by phone model, with binwidths of 8.33 ms. All phone models show 60 Hz RT discretization.

Figure 4

Plot of effects of repeated exposure to CogGauge assessments on response time (RT). 1-Back, logical reasoning, and math processing RT followed an exponential decay relative to sessions played with half-lives of 0.67, 1.5, and 2.6 sessions, respectively.

Figure 5

Timeline of three representative participant’s mental acuity, sleep data, and model fit. Mental acuity results are shown as black dots, sleep periods are shown in gray, the group model fit is shown as a red line, and the individual fits are shown as blue lines.

Figure 6

Unified model of performance fit as measured by the ratio of the root mean squared error (RMSE) of the unified model fit to a horizontal line fit, representing mental acuity independent of sleep effects. Group model is represented by a vertical line.