Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Jun 21;10(6):giab044.
doi: 10.1093/gigascience/giab044.

Multi-modal data collection for measuring health, behavior, and living environment of large-scale participant cohorts

Congyu Wu  1 , Hagen Fritz  2 , Sepehr Bastami  2 , Juan P Maestre  2 , Edison Thomaz  3 , Christine Julien  3 , Darla M Castelli  4 , Kaya de Barbaro  1 , Sarah Kate Bearman  5 , Gabriella M Harari  6 , R Cameron Craddock  7 , Kerry A Kinney  2 , Samuel D Gosling  1   8 , David M Schnyer  1 , Zoltan Nagy  2
Affiliations

Multi-modal data collection for measuring health, behavior, and living environment of large-scale participant cohorts

Congyu Wu et al. Gigascience. .

Abstract

Background: As mobile technologies become ever more sensor-rich, portable, and ubiquitous, data captured by smart devices are lending rich insights into users' daily lives with unprecedented comprehensiveness and ecological validity. A number of human-subject studies have been conducted to examine the use of mobile sensing to uncover individual behavioral patterns and health outcomes, yet minimal attention has been placed on measuring living environments together with other human-centered sensing data. Moreover, the participant sample size in most existing studies falls well below a few hundred, leaving questions open about the reliability of findings on the relations between mobile sensing signals and human outcomes.

Results: To address these limitations, we developed a home environment sensor kit for continuous indoor air quality tracking and deployed it in conjunction with smartphones, Fitbits, and ecological momentary assessments in a cohort study of up to 1,584 college student participants per data type for 3 weeks. We propose a conceptual framework that systematically organizes human-centric data modalities by their temporal coverage and spatial freedom. Then we report our study procedure, technologies and methods deployed, and descriptive statistics of the collected data that reflect the participants' mood, sleep, behavior, and living environment.

Conclusions: We were able to collect from a large participant cohort satisfactorily complete multi-modal sensing and survey data in terms of both data continuity and participant adherence. Our novel data and conceptual development provide important guidance for data collection and hypothesis generation in future human-centered sensing studies.

Keywords: BEVO Beacon; Fitbit; college students; ecological momentary assessment; health; human-centered computing; multi-modal sensing; smartphone.

PubMed Disclaimer

Figures

Figure 1:
Figure 1:
Human-centric data modality framework.
Figure 2:
Figure 2:
Building EnVironment and Occupancy (BEVO) Beacon.
Figure 3:
Figure 3:
Data collected from the smartphone, Fitbit, and BEVO Beacon of an example participant during a given day (30 March 2019). Plotted data modalities are EMA (questions and answers shown against phone image background on top), GPS (clustered significant places and an example movement trajectory shown in maps on top and as vertical bands), accelerometry (black dots), screen activity (short grey bands), heart rate (BPM), calories spent in the past hour (Calories), steps taken in the past hour (StepTotal), home relative humidity (RH), home temperature in degrees Fahrenheit (TF), home particulate matter in μg/m3 (PM), and home TVOC (TVOC).
Figure 4:
Figure 4:
Completeness of 4 types of data collected from participants in the Fall and Spring deployments combined: (A) daily EMA (1,482 participants submitted data); (B) smartphone sensing, GPS data shown as example (1,539 participants); (C) Fitbit (36 participants); and (D) BEVO Beacon (9 participants).
See this image and copyright information in PMC

References

    1. Harari GM, Lane ND, Wang R, et al. . Using smartphones to collect behavioral data in psychological science: Opportunities, practical considerations, and challenges. Perspect Psychol Sci. 2016;11(6):838–54. - PMC - PubMed
    1. Aharony N, Pan W, Ip C, et al. . Social fMRI: Investigating and shaping social mechanisms in the real world. Pervasive Mob Comput. 2011;7(6):643–59.
    1. Wang R, Chen F, Chen Z, et al. . StudentLife: Assessing mental health, academic performance and behavioral trends of college students using smartphones. In: Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing. ACM; 2014:3–14.
    1. Stopczynski A, Sekara V, Sapiezynski P, et al. . Measuring large-scale social networks with high resolution. PLoS One. 2014;9(4):e95978. - PMC - PubMed
    1. Purta R, Mattingly S, Song L, et al. . Experiences measuring sleep and physical activity patterns across a large college cohort with fitbits. In: Proceedings of the 2016 ACM International Symposium on Wearable Computers; 2016:28–35.

Publication types