Automated Cough Assessment on a Mobile Platform

Abstract

The development of an Automated System for Asthma Monitoring (ADAM) is described. This consists of a consumer electronics mobile platform running a custom application. The application acquires an audio signal from an external user-worn microphone connected to the device analog-to-digital converter (microphone input). This signal is processed to determine the presence or absence of cough sounds. Symptom tallies and raw audio waveforms are recorded and made easily accessible for later review by a healthcare provider. The symptom detection algorithm is based upon standard speech recognition and machine learning paradigms and consists of an audio feature extraction step followed by a Hidden Markov Model based Viterbi decoder that has been trained on a large database of audio examples from a variety of subjects. Multiple Hidden Markov Model topologies and orders are studied. Performance of the recognizer is presented in terms of the sensitivity and the rate of false alarm as determined in a cross-validation test.

Figure 1

Representative audio waveforms from database. The top trace shows a long segment of silence followed by a cough. The bottom trace shows a segment of background noise. These are 16-bit PCM.wav files at a sampling rate of 11025 Hz.

Figure 2

32-point Mel-filterbank used in the feature extraction step.

Figure 3

Illustrative example of the audio features (MFCCs and log-energy) seen by the classifier.

Figure 4

Image of the transition map of the composite HMM. The matrix has a block structure with each block corresponding to one of the sound classes, background, cough, silence. The intensity of each pixel in this image corresponds to the log-probability p _ij for a token to pass from the row to the column. The off-diagonal blocks correspond to transitions between the various sound classes.

Figure 5

The ergodic/connected HMM topology (a) and the left-to-right HMM topology (b). The connected topology allows a transition from one state to any other state. The left-to-right topology only allows a transition to the same state or to the next state up.

Figure 6

Grammar network used for constructing a composite HMM from silence, cough, and background HMMs. This parallel structure is the most simple and freest way to combine the individual HMMs. The gray boxes represent dummy states that route the outputs to the inputs (and, importantly, do not cost a time unit).

Figure 7

Example Viterbi decoding pass for audio containing coughs. In this case the background model had 6 states with 60 mixtures per state while the cough model had 5 states with 60 mixtures per state with a left-to-right topology. The horizontal axis represents time (at the frame rate) while the vertical axis is an integer of the state index. Gray marks indicate states associated with the silence and background models. Black marks indicate states associated with the cough model.

Figure 8

Screenshots of the ADAM application showing live capture and processing of audio data. The top traces are time-domain audio waveforms and the bottom traces indicate the presence of coughs. The left screenshot shows a sequence of coughs being recognized while the right screenshot shows some example speech being discounted. This is a demonstration mode used for debugging that is normally inaccessible to patients; hence the interface is somewhat rough. The total length of audio represented is 6 seconds in each screenshot. The middle number at the upper-left is the actual cough count which is determined by finding the instances where the last cough state is followed by the first state of any model in the grammar.

Algorithm 1