App-based symptom tracking to optimize SARS-CoV-2 testing strategy using machine learning

Abstract

Background: Tests are scarce resources, especially in low and middle-income countries, and the optimization of testing programs during a pandemic is critical for the effectiveness of the disease control. Hence, we aim to use the combination of symptoms to build a predictive model as a screening tool to identify people and areas with a higher risk of SARS-CoV-2 infection to be prioritized for testing.

Materials and methods: We performed a retrospective analysis of individuals registered in "Dados do Bem," a Brazilian app-based symptom tracker. We applied machine learning techniques and provided a SARS-CoV-2 infection risk map of Rio de Janeiro city.

Results: From April 28 to July 16, 2020, 337,435 individuals registered their symptoms through the app. Of these, 49,721 participants were tested for SARS-CoV-2 infection, being 5,888 (11.8%) positive. Among self-reported symptoms, loss of smell (OR[95%CI]: 4.6 [4.4-4.9]), fever (2.6 [2.5-2.8]), and shortness of breath (2.1 [1.6-2.7]) were independently associated with SARS-CoV-2 infection. Our final model obtained a competitive performance, with only 7% of false-negative users predicted as negatives (NPV = 0.93). The model was incorporated by the "Dados do Bem" app aiming to prioritize users for testing. We developed an external validation in the city of Rio de Janeiro. We found that the proportion of positive results increased significantly from 14.9% (before using our model) to 18.1% (after the model).

Conclusions: Our results showed that the combination of symptoms might predict SARS-Cov-2 infection and, therefore, can be used as a tool by decision-makers to refine testing and disease control strategies.

Fig 1. Association between symptoms and the SARS-CoV-2 infection.

The Odds Ratio (OR) with 95% confidence intervals using logistic regression models for each feature was adjusted by age and gender.

Fig 2. Boxplots representing the Matthews Correlation Coefficients (MCC) of each model and balancing technique combination (points) for all methods.

Boxplots represent the distribution of MCC values for each model and balancing technique combination. The higher the MCC value, the better the model.

Fig 3. Confusion matrix and performance metrics of the final model.

Fig 4

SARS-CoV-2 infection risk map (grid) of Rio de Janeiro state, displaying the city of Rio de Janeiro (capital, left) and the city of Niteroi (right). The grid risk considered the proportion of potential positive infection (observed test results + estimated from the prediction model) for each grid (400mx400m area). The risk groups were obtained as very low (<17%), low (≥17% and <33%), medium (≥33% and <48%), high (≥48 and <63%), and very high (≥63%). The map shows the distribution of risks as of June 10, 2020. Map created by Dados do Bem app using OpenStreetMap @ 2020Mapbox.

Fig 5. Risk map of two neighborhoods of Rio de Janeiro (Rocinha–very high risk and Ipanema–low risk).

Map created by Dados do Bem app using OpenStreetMap @ 2020Mapbox.