Detection of Common Respiratory Infections, Including COVID-19, Using Consumer Wearable Devices in Health Care Workers: Prospective Model Validation Study

Abstract

Background: The early detection of respiratory infections could improve responses against outbreaks. Wearable devices can provide insights into health and well-being using longitudinal physiological signals.

Objective: The purpose of this study was to prospectively evaluate the performance of a consumer wearable physiology-based respiratory infection detection algorithm in health care workers.

Methods: In this study, we evaluated the performance of a previously developed system to predict the presence of COVID-19 or other upper respiratory infections. The system generates real-time alerts using physiological signals recorded from a smartwatch. Resting heart rate, respiratory rate, and heart rate variability measured during the sleeping period were used for prediction. After baseline recordings, when participants received a notification from the system, they were required to undergo testing at a Northwell Health System site. Participants were asked to self-report any positive tests during the study. The accuracy of model prediction was evaluated using respiratory infection results (laboratory results or self-reports), and postnotification surveys were used to evaluate potential confounding factors.

Results: A total of 577 participants from Northwell Health in New York were enrolled in the study between January 6, 2022, and July 20, 2022. Of these, 470 successfully completed the study, 89 did not provide sufficient physiological data to receive any prediction from the model, and 18 dropped out. Out of the 470 participants who completed the study and wore the smartwatch as required for the 16-week study duration, the algorithm generated 665 positive alerts, of which 153 (23.0%) were not acted upon to undergo testing for respiratory viruses. Across the 512 instances of positive alerts that involved a respiratory viral panel test, 63 had confirmed respiratory infection results (ie, COVID-19 or other respiratory infections detected using a polymerase chain reaction or home test) and the remaining 449 had negative upper respiratory infection test results. Across all cases, the estimated false-positive rate based on predictions per day was 2%, and the positive-predictive value ranged from 4% to 10% in this specific population, with an observed incidence rate of 198 cases per week per 100,000. Detailed examination of questionnaires filled out after receiving a positive alert revealed that physical or emotional stress events, such as intense exercise, poor sleep, stress, and excessive alcohol consumption, could cause a false-positive result.

Conclusions: The real-time alerting system provides advance warning on respiratory viral infections as well as other physical or emotional stress events that could lead to physiological signal changes. This study showed the potential of wearables with embedded alerting systems to provide information on wellness measures.

Keywords: COVID detection; COVID-19; algorithm; emotional stress; health; health care worker; infection; physical stress; physiology; prediction; respiratory infection; respiratory virus; respiratory virus detection; wearable; wearable device; well-being.

Figure 1

Positive algorithm detection rate of polymerase chain reaction (PCR)-confirmed COVID-19 relative to the length of the predictive window. The predictive window was the time window (days) between the test date and the date for an alert to be accepted as a correct detection (ie, predictive window=8; alert generated within 8 days prior to the test date counted as a correct detection). A wider predictive window was associated with a higher detection rate of the algorithm for PCR-confirmed COVID-19 cases.

Figure 2

Study protocol. Day 0 to day 18: onboarding, baseline measurement, and issuing new devices. Upon receiving a positive alert from day 19 onward, participants were instructed to take the respiratory viral panel (RVP) test as well as fill out follow-up questionnaires the next day after receiving an alert. Daily symptom questionnaires were filled out throughout the study. Participants only received notifications for positive alerts. The algorithm required at least 3 nights of data in a rolling window of the past 5 nights to be able to generate predictions. In total, 89 participants in this study did not adhere to the study guidelines, and the algorithm could not generate any prediction owing to infrequent use of the smartwatch.

Figure 3

Wear time and algorithm predictions during the study. (A) Distribution of signal coverage (percentage of days participants wore the watch to bed during the study). (B) Distribution of the average daily smartwatch wear time (hours) during the study across all participants. (C) Number of predictions received by each participant during the study. (D) Distribution of time when the first algorithm prediction was generated after establishing baseline.

Figure 4

Algorithm alert flowchart following the STROBE (Strengthening the Reporting of Observational Studies in Epidemiology) reporting guidelines. Participants only received a notification to undergo testing if the algorithm generated a positive alert. Participants were instructed to report any positive upper respiratory infection test results they received for reasons other than positive alerts from our algorithm.

Figure 5

Associations of test results with symptoms across participants with positive and negative test results who received an alert. Bar plots show the percentage of positive tests associated with each symptom. Symptoms were considered within a window of 10 days before the positive test result to 10 days after the test result. (A) Participants with positive COVID-19 polymerase chain reaction (PCR) test results. (B) Participants with positive results for respiratory viruses confirmed with PCR or home tests versus participants who received an alert with negative PCR test results. RVP: respiratory viral panel.