Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2025 Apr 3:27:e63090.
doi: 10.2196/63090.

Investigating Measurement Equivalence of Smartphone Sensor-Based Assessments: Remote, Digital, Bring-Your-Own-Device Study

Lito Kriara  1 , Frank Dondelinger  1 , Luca Capezzuto  1 , Corrado Bernasconi  1 , Florian Lipsmeier  1 , Adriano Galati  1 , Michael Lindemann  1
Affiliations

Investigating Measurement Equivalence of Smartphone Sensor-Based Assessments: Remote, Digital, Bring-Your-Own-Device Study

Lito Kriara et al. J Med Internet Res. .

Abstract

Background: Floodlight Open is a global, open-access, fully remote, digital-only study designed to understand the drivers and barriers in deployment and persistence of use of a smartphone app for measuring functional impairment in a naturalistic setting and broad study population.

Objective: This study aims to assess measurement equivalence properties of the Floodlight Open app across operating system (OS) platforms, OS versions, and smartphone device models.

Methods: Floodlight Open enrolled adult participants with and without self-declared multiple sclerosis (MS). The study used the Floodlight Open app, a "bring-your-own-device" (BYOD) solution that remotely measured MS-related functional ability via smartphone sensor-based active tests. Measurement equivalence was assessed in all evaluable participants by comparing the performance on the 6 active tests (ie, tests requiring active input from the user) included in the app across OS platforms (iOS vs Android), OS versions (iOS versions 11-15 and separately Android versions 8-10; comparing each OS version with the other OS versions pooled together), and device models (comparing each device model with all remaining device models pooled together). The tests in scope were Information Processing Speed, Information Processing Speed Digit-Digit (measuring reaction speed), Pinching Test (PT), Static Balance Test, U-Turn Test, and 2-Minute Walk Test. Group differences were assessed by permutation test for the mean difference after adjusting for age, sex, and self-declared MS disease status.

Results: Overall, 1976 participants using 206 different device models were included in the analysis. Differences in test performance between subgroups were very small or small, with percent differences generally being ≤5% on the Information Processing Speed, Information Processing Speed Digit-Digit, U-Turn Test, and 2-Minute Walk Test; <20% on the PT; and <30% on the Static Balance Test. No statistically significant differences were observed between OS platforms other than on the PT (P<.001). Similarly, differences across iOS or Android versions were nonsignificant after correcting for multiple comparisons using false discovery rate correction (all adjusted P>.05). Comparing the different device models revealed a statistically significant difference only on the PT for 4 out of 17 models (adjusted P≤.001-.03).

Conclusions: Consistent with the hypothesis that smartphone sensor-based measurements obtained with different devices are equivalent, this study showed no evidence of a systematic lack of measurement equivalence across OS platforms, OS versions, and device models on 6 active tests included in the Floodlight Open app. These results are compatible with the use of smartphone-based tests in a bring-your-own-device setting, but more formal tests of equivalence would be needed.

Keywords: Floodlight Open; autoimmune disease; balance; cognition; device equivalence; digital assessment; digital biomarker; digital health; equivalence; gait; hand motor function; mHealth; mobile health; mobile phone; motor; multiple sclerosis; sensors; smartphone; upper extremity function; variability; wearable electronic devices.

PubMed Disclaimer

Conflict of interest statement

Conflicts of Interest: LK, LC, FL, and AG are employees of F. Hoffmann-La Roche Ltd. FD was an employee of F. Hoffmann-La Roche Ltd. during the completion of the work related to this manuscript. FD is now an employee of Novartis (Basel, Switzerland), which was not in any way associated with this study. CB was a contractor for F. Hoffmann-La Roche Ltd. during the completion of the work related to this manuscript. CB is now with Limites Medical Research Ltd., which was not in any way associated with this study. ML is a consultant for F. Hoffmann-La Roche Ltd via Inovigate.

Figures

Figure 1
Figure 1
Participant deposition and the devices they used.
Figure 2
Figure 2
Measurement equivalence by operating system (OS) platform. Permutation testing showed no evidence of a systematic lack of equivalence between the 2 OS platforms—iOS and Android. A small but statistically significant difference was only observed on the PT. Differences between groups as well as effect sizes are provided in Table 3. Brackets indicate the sample size. ***P<.001. IPS: Information Processing Speed; IPS DD: Information Processing Speed Digit-Digit; PT: Pinching Test; SBT: Static Balance Test; UTT: U-Turn Test; 2MWT: 2-Minute Walk Test.
Figure 3
Figure 3
Measurement equivalence by (A) iOS version and (B) Android version. Permutation testing revealed no evidence of a systematic lack of equivalence. No statistically significant differences were observed across operating system versions after correcting for multiple comparisons (all Padjusted>.05). Absolute and percent differences as well as effect sizes are provided in Tables 4 and 5. Brackets indicate the sample size. IPS: Information Processing Speed; IPS DD: Information Processing Speed Digit-Digit; PT: Pinching Test; SBT: Static Balance Test; UTT: U-Turn Test; 2MWT: 2-Minute Walk Test.
See this image and copyright information in PMC

Similar articles

See all similar articles

References

    1. Reich DS, Lucchinetti CF, Calabresi PA. Multiple sclerosis. N Engl J Med. 2018;378(2):169–180. doi: 10.1056/NEJMra1401483. https://europepmc.org/abstract/MED/29320652 - DOI - PMC - PubMed
    1. Walton C, King R, Rechtman L, Kaye W, Leray E, Marrie RA, Robertson N, La Rocca N, Uitdehaag B, van der Mei I, Wallin M, Helme A, Angood Napier C, Rijke N, Baneke P. Rising prevalence of multiple sclerosis worldwide: insights from the Atlas of MS, third edition. Mult Scler. 2020;26(14):1816–1821. doi: 10.1177/1352458520970841. https://journals.sagepub.com/doi/10.1177/1352458520970841 - DOI - DOI - PMC - PubMed
    1. Hobart J, Bowen A, Pepper G, Crofts H, Eberhard L, Berger T, Boyko A, Boz C, Butzkueven H, Celius EG, Drulovic J, Flores J, Horáková D, Lebrun-Frénay C, Marrie RA, Overell J, Piehl F, Rasmussen PV, Sá MJ, Sîrbu CA, Skromne E, Torkildsen Ø, van Pesch V, Vollmer T, Zakaria M, Ziemssen T, Giovannoni G. International consensus on quality standards for brain health-focused care in multiple sclerosis. Mult Scler. 2019;25(13):1809–1818. doi: 10.1177/1352458518809326. https://journals.sagepub.com/doi/10.1177/1352458518809326 - DOI - DOI - PMC - PubMed
    1. Rae-Grant A, Bennett A, Sanders AE, Phipps M, Cheng E, Bever C. Quality improvement in neurology: multiple sclerosis quality measures: executive summary. Neurology. 2015;85(21):1904–1908. doi: 10.1212/WNL.0000000000001965. https://europepmc.org/abstract/MED/26333795 WNL.0000000000001965 - DOI - PMC - PubMed
    1. Clay I. Impact of digital technologies on novel endpoint capture in clinical trials. Clin Pharmacol Ther. 2017;102(6):912–913. doi: 10.1002/cpt.866. - DOI - PubMed

LinkOut - more resources