Project Achoo: A Practical Model and Application for COVID-19 Detection From Recordings of Breath, Voice, and Cough

Abstract

The COVID-19 pandemic created significant interest and demand for infection detection and monitoring solutions. In this paper, we propose a machine learning method to quickly detect COVID-19 using audio recordings made on consumer devices. The approach combines signal processing and noise removal methods with an ensemble of fine-tuned deep learning networks and enables COVID detection on coughs. We have also developed and deployed a mobile application that uses a symptoms checker together with voice, breath, and cough signals to detect COVID-19 infection. The application showed robust performance on both openly sourced datasets and the noisy data collected during beta testing by the end users.

Keywords: Acoustic signal processing; Big Data applications; biomedical informatics; machine learning; public heathcare; signal detection.

Fig. 1.

The diagram of the entire pipeline of our solution.

Fig. 2.

Datasets used in our work. Models were trained on the Covid19-Cough dataset and fine-tuned on the Covid19-Cough and the part of the “Hospital” dataset. Red dotted boxes indicate the data that was used in some way to change the weights of our models. Green dotted boxes indicate those parts of datasets that were used for stacking and/or calculating the target metrics.

Fig. 3.

The ROC curves for predictions of models on 10 distinct test datasets from Covid19-cough. Blue depicts the mean ROC curve calculated by averaging false positive rates and true positive rates of individual predictors. The ROC curve on the left was built for deep neural networks; on the right – for gradient boosting.

Fig. 4.

Histogram depicts how output probabilities of the Ensemble (Variant II) are distributed. In green – all cases; in red – cases where users reports absence of a respiratory disease; in blue – cases where users reports presence of a respiratory disease.

Fig. 5.

Receiver operating characteristic curve of the cough validator.

Fig. 6.

Distribution of cough validator model outputs for 6480 recordings collected by the mobile application over several days following release.