Few-Shot User-Adaptable Radar-Based Breath Signal Sensing

Abstract

Vital signs estimation provides valuable information about an individual's overall health status. Gathering such information usually requires wearable devices or privacy-invasive settings. In this work, we propose a radar-based user-adaptable solution for respiratory signal prediction while sitting at an office desk. Such an approach leads to a contact-free, privacy-friendly, and easily adaptable system with little reference training data. Data from 24 subjects are preprocessed to extract respiration information using a 60 GHz frequency-modulated continuous wave radar. With few training examples, episodic optimization-based learning allows for generalization to new individuals. Episodically, a convolutional variational autoencoder learns how to map the processed radar data to a reference signal, generating a constrained latent space to the central respiration frequency. Moreover, autocorrelation over recorded radar data time assesses the information corruption due to subject motions. The model learning procedure and breathing prediction are adjusted by exploiting the motion corruption level. Thanks to the episodic acquired knowledge, the model requires an adaptation time of less than one and two seconds for one to five training examples, respectively. The suggested approach represents a novel, quickly adaptable, non-contact alternative for office settings with little user motion.

Keywords: FMCW; artificial neural networks; autocorrelation; few-shot learning; meta-learning; radar; respiration signal; signal processing; variational autoencoder; vital sign sensing.

Figure 1

For each learning episode, a training subject is randomly sampled. For each training shot, the radar phase information is mapped to the reference belt signal (ref.) via a C-VAE. Through a dense layer, the ANN also tries to regress the extracted respiration $Fc$, learning from the ideal belt $Fc$. The latent space mapping is thus constrained to the $Fc$, whose estimate is also used in the prediction phase.

Figure 2

The diagram shows the main steps of the implementation. For a chosen scenario (room and user), several data sessions with synchronized radar and a reference respiration belt are collected. For multi-output ANN, the labels consist of belt reference signals and the central breath frequencies, estimated from the pure belt reference. The data from fourteen users are then used to train an ANN episodically using Meta-L, while the data from the remaining ten users are solely used for testing.

Figure 3

The BGT60TR13 radar system (a) delivers filtered, mixed, and digitized information from each Rx channel. The BGT60TR13C radar (b) is mounted on top of the evaluation board.

Figure 4

Recording Setup. A synchronized radar system and respiration belt are used to collect 10 30-second sessions per user and distance. The distance ranges used in data collection (up to 30 or 40 cm), refer to the distance between the chest and the radar board.

Figure 5

Preprocessing pipeline. First, the phase information is unwrapped from the raw radar data. The respiration signal and $Fc$ are then estimated by Meta-L, exploiting only in the training phase the data collected with the respiration belt.

Figure 6

Lines in yellow indicate the defined range bin limits and, in red, the detected maximum bin per frame. Range plotting is generated after clutter removal. In (a), the subject did not move much during the session. In (b), the range limits vary according to the user’s distance from the radar board.

Figure 7

Band-pass bi-quadratic filter. The diagram (a) depicts the linear flow of the biquad filter, where the output $O\left(n\right)$ at time instant n is determined by the two previous input I and output O values. Instead, a gain vs. frequency plot of a biquad band-pass filter obtained for a Q of $\sqrt{2}$ and $fs$ of 20, over an $Fc$ of 0.33 Hz, is shown as a reference in (b).

Figure 8

Example of sliding window generation for instant bpm estimation on a recorded session The radar signal has been filtered using the ideal belt, $Fc$. The radar, as opposed to the belt, is not connected to the user during recordings, but to the desk. This results in the local shift of signal breathing peaks due to the millimeter movements of the user. The window (in purple in the plot) is shown paler on the two peaks closest to the calculated peaks’ mean distance. It is also possible to notice some slight corruption at the beginning of the session due to user motion.

Figure 9

Comparison of instantaneous bpm between respiration belt and radar (with ideal $Fc$) for a recording session. The x-axis corresponds to the difference between the number of frames in the session and the sliding window length. The radar signal corruption flag variable is plotted in green. At the beginning of the session, the radar signal is motion-corrupted (as shown in Figure 8) and thus does not lead to a reliable bpm. On the other hand, for the workplace use case, the reference belt signal is more robust to motion. In this case, the motion performed was the movement of the hands toward the desk.

Figure 10

Two-component t-SNE representation of the Breath Meta-Dataset radar data. The circles represent the training users, while the crosses represent the testing users for the Meta-L. No user-specific feature clusters are visible under the t-SNE assumptions. The t-SNE was obtained with a perplexity of 20 and 7000 iterations [42].

Figure 11

Graphical representation of single-episode learning with C-VAE. The unwrapped radar phase is mapped to the respiration belt signal using the signal reconstruction term. The regularization term makes the latent space closer to a standard multivariate normal distribution. $Fc$ regression allows the parameterization to depend on the respiration signal.

Figure 12

Chosen C-VAE topology. The latent space representation is constrained by both the reconstruction of x with respect to the $x_{belt}$ reference and the ideal $Fc$ of breathing y. The decoder layers are an up-sampled mirror version of the encoder layers.

Figure 13

Examples of latent space generation. Examples of radar phase input (a) and generated latent spaces (b), size 32, are shown. The latent spaces are obtained after the model generalization training. Each $8x8$ representation consists of the mean values $\mu$ and the standard deviations $\sigma$. Starting from the top of the representations toward the right, the first 32 pixels represent $\mu$ values, while the last 32 are those of $\sigma$.

Figure 14

MAML $2^{nd}$ 1–shot experiment, Box Plots. Learning trends of Meta-L, box plots versus episodes (evaluation loop) for the Breath Meta-Dataset. The box in (a) depicts the trend for users in the training set (${\mathcal{T}}_r$ tasks). In (b), the trend for the users of the test set (${\mathcal{T}}_v$ tasks)is shown. The box’s mid-line represents the median value, while the little green triangle represents the mean.

Figure 15

MAML $2^{nd}$ 1–shot experiment histograms for the first (a) and last (b) set of 300 episodes. The box plots in the topmost plots also contain outliers as small circles outside the whiskers. The mid-plots show an approximation to the Gaussian distribution. The lower plots show the true histograms, which do not underlie a Gaussian distribution. The q1 and q3 represent the first and third quartiles, respectively.

Figure 16

Loss ($L^{*}$ ) as a function of the number of detected breathing spikes over the 30 s sessions for the 10 test users. The base of the box plots with non-uniform ranges was chosen so as to have at least 4 examples for the least common classes (1–4 and 12–14). The upper plot is obtained by fitting the 1–shot Meta-L model (a) to new users, while the middle and lower plots are obtained by 5– (b) and 10– (c) shots adaptation, respectively. For the first two plots, the circles that lie outside the box plots whiskers represent the outliers. Plot (c) shows no visible outliers.

Figure 17

Standard prediction examples obtained post 1–shot test user-adaptation with MAML $2^{nd}$. The top plots show the prediction ${\widehat{x}}^{*}$ versus the respiration belt reference, while the bottom plots display the estimated bpm and corruption flag. Legends, which also apply to the plots on the right, are placed in the plots on the left. An example of optimal prediction with radar information characterized by little motion corruption is shown in (a). The respiration signal is recovered even in the presence of some corruption, as in (b), thanks to the $L^{*}$ formulation.

Figure 18

Edge prediction examples obtained post 1–shot test user-adaptation with MAML $2^{nd}$. The top plots show the prediction ${\widehat{x}}^{*}$ versus the respiration belt reference, while the bottom plots display the estimated bpm and corruption flag. Legends, which also apply to the plots on the right, are placed in the plots on the left. In (a), there are six visible peaks in the belt signal (blue), while in (b) there are thirteen peaks. In these examples, the algorithm performs less well than in standard cases. This is mainly due to the lack of edge data as prior knowledge during episodic learning. In the bpm estimation in the example (a), a shorter estimate can be seen for the belt than for radar. This is due to the computation of two distinct windows between radar and belt, as explained in Section 3.6.