Evaluation of Wearable Digital Devices in a Phase I Clinical Trial

Abstract

We assessed the performance of two US Food and Drug Administration (FDA) 510(k)-cleared wearable digital devices and the operational feasibility of deploying them to augment data collection in a 10-day residential phase I clinical trial. The Phillips Actiwatch Spectrum Pro (Actiwatch) was used to assess mobility and sleep, and the Vitalconnect HealthPatch MD (HealthPatch) was used for monitoring heart rate (HR), respiratory rate (RR), and surface skin temperature (ST). We measured data collection rates, compared device readouts with anticipated readings and conventional in-clinic measures, investigated data limitations, and assessed user acceptability. Six of nine study participants consented; completeness of data collection was adequate (> 90% for four of six subjects). A good correlation was observed between the HealthPatch device derived and in-clinic measures for HR (Pearson r = 0.71; P = 2.2e-16) but this was poor for RR (r = 0.08; P = 0.44) and ST (r = 0.14; P = 0.14). Manual review of electrocardiogram strips recorded during reported episodes of tachycardia > 180 beats/min showed that these were artefacts. The HealthPatch was judged to be not fit-for-purpose because of artefacts and the need for time-consuming manual review. The Actiwatch device was suitable for monitoring mobility, collecting derived sleep data, and facilitating the interpretation of vital sign data. These results suggest the need for fit-for-purpose evaluation of wearable devices prior to their deployment in drug development studies.

Figure 1

The comparison of in‐clinic and wearable measurements for heart rate (HR), respiratory rate (RR), and skin temperature (ST) by correlation, regression (a−c), and Bland‐Altman (d−f) analyses. The solid lines in a, b, and c are the line y = x, which would be expected if concordance were perfect; the dotted line is the actual regression line. In c there is no solid line because ST is significantly lower than core temperature. The Bland‐Altman analysis results are presented in d, e, and f. The black dotted line is the line y = 0 indicating no mean difference between the two measures. The red dotted line is the mean difference line based on the actual data. f ST there is no black dotted line because ST is significantly lower than core temperature. a and d show the analyses for HR, b and e shows RR, c and f shows ST. Points are color‐coded with a unique color for each subject.

Figure 2

The aggregate diurnal patterns for heart rate (a), respiratory rate (b), skin temperature (c), and activity counts (d).

Figure 3

The histogram depicting the range of value distribution for heart rate (a), respiratory rate (b), and skin temperature (c).

Figure 4

Visualization of heart rate (HR) (a, b) and respiratory rate (c, d) temporal patterns in the context of wake–sleep data (e, f), and subject mobility as assessed by activity counts (g, h) for subjects 58001_0003 (a, c, e, and g) and subject 58001_0011 (b, d, f, and h). bpm, beats/min.

Figure 5

Visual examination of the echocardiogram (ECG) waveforms for episodes with reported heart rates of 44 bpm (a), 79 bpm (b), 203 bpm (c), and 193 bpm (d), bpm, beats/min.