Reliability of an all-in-one wearable sensor for continuous vital signs monitoring in high-risk patients: the NIGHTINGALE clinical validation study

Abstract

Continuous vital signs monitoring with wearable systems may improve early recognition of patient deterioration on hospital wards. The objective of this study was to determine whether the wearable Checkpoint Cardio's CPC12S, can accurately measure heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2), blood pressure (BP) and temperature continuously. In an observational multicenter method comparison study of 70 high-risk surgical patients admitted to high-dependency wards; HR, RR, SpO2, BP and temperature were simultaneously measured with the CPC12S system and with ICU-grade monitoring systems in four European hospitals. Outcome measures were bias and 95% limits of agreement (LoA). Clinical accuracy was assessed with Clarke Error Grid analyses for HR and RR. A total of 3,212 h of vital signs data (on average 26 h per patient) were analyzed. For HR, bias (95% LoA) of the pooled analysis was 0.0 (-3.5 to 3.4), for RR 1.5 (-3.7 to 7.5) and for SpO2 0.4 (-3.1 to 4.0). The CPC12S system overestimated BP, with a bias of 8.9 and wide LoA (-23.3 to 41.2). Temperature was underestimated with a bias of -0.6 and LoA of -1.7 to 0.6. Clarke Error Grid analyses showed that adequate treatment decisions regarding changes in HR and RR would have been made in 99.2% and 92.0% of cases respectively. The CPC12S system showed high accuracy for measurements of HR. The accuracy of RR, SpO2 were slightly overestimated and core temperature underestimated, with LoA outside the predefined clinical acceptable range. The accuracy of BP was unacceptably low.

Keywords: Clinical deterioration; Continuous monitoring; Remote monitoring; Vital signs; Wearable device.

Fig. 1

The CPC12S Nightingale multiparameter monitoring system (Checkpoint Cardio Ltd, Bulgaria). The wearable sensor attached with two electrodes on the chest measures ECG and HR. RR is derived using impedance pneumography. The ear sensor measures photoplethysmogram to determine SpO2. BP is derived from pulse transit time (PTT) using signals from both PPG and ECG. Temperature is measured by a thermistor placed in the axilla

Fig. 2

(a) Bland-Altman plot of the pooled analysis of all heart rate measurements with few (white) to many (dark red) measurement pairs. The dashed black line corresponds to the limits of agreement from the Bland-Altman method, and the dashed red line from mixed effects models respectively. Bias is shown as a black line. (b) Bland-Altman plot of the pooled analysis of all respiratory rate measurements with few (white) to many (dark red) measurement pairs. The dashed black line corresponds to the limits of agreement from the Bland-Altman method, and the dashed red line from mixed effects models respectively. Bias is shown as a black line. (c) Bland-Altman plot of the pooled analysis of all oxygen saturation (SpO2) measurements with few (white) to many (dark red) measurement pairs. The dashed black line corresponds to the limits of agreement from the Bland-Altman method, and the dashed red line from mixed effects models respectively. Bias is shown as black line. (d) Bland-Altman plot of the pooled analysis of all mean arterial pressure (MAP) measurements with few (white) to many (dark red) measurement pairs. The dashed black line corresponds to the limits of agreement from the Bland-Altman method, and the dashed red line from mixed effects models respectively. Bias is shown as a black line. (e) Bland-Altman plot of the pooled analysis of all temperature measurements with few (white) to many (dark red) measurement pairs. The dashed black line corresponds to the limits of agreement from the Bland-Altman method, and the dashed red line from mixed effects models respectively. Bias is shown as a black line

Fig. 3

(a and b) Clarke Error Grid analysis to quantify clinical accuracy of heart rate measurements (a; left panel) and respiratory rate measurements (b; right panel) with the CPC12S system compared with the reference standard. The colored dots are measurement pairs superimposed on the error grid boundaries, where the color intensity is proportional to the number of observations. Region A shows points within 20% of the reference monitor; region B contains points outside 20% of the reference, but not leading to unnecessary treatment. Region C contains points leading to unnecessary treatment, region D indicates a potentially dangerous failure to detect e.g., bradycardia or tachycardia, and region E represents points where events are confused (e.g., bradycardia with tachycardia) in case of heart rate measurements

Fig. 4

(a) Scatterplot comparing measurements of the pooled analysis of oxygen saturation with few (white) to many (dark red) measurement pairs from the CPC12S system and reference systems. (b) Scatterplot comparing measurements of the pooled analysis of mean arterial pressure with few (white) to many (dark red) measurement pairs from the CPC12S system and reference systems. (c) Scatterplot comparing measurements of the pooled analysis of temperature with few (white) to many (dark red) measurement pairs from the CPC12S system and reference systems

Fig. 5

Example of a patient that is being continuously monitored for more than four days with the CPC12S system (red) and reference standard UMC Utrecht (blue). From top to bottom, the panels show heart rate (HR), respiratory rate (RR), oxygen saturation (SpO2), mean arterial pressure (MAP) and temperature measurements. Three clinical events occurring are marked in the lowest panel. This example shows unfiltered data from both systems