Remote monitoring of cardiorespiratory signals from a hovering unmanned aerial vehicle

Abstract

Background: Remote physiological measurement might be very useful for biomedical diagnostics and monitoring. This study presents an efficient method for remotely measuring heart rate and respiratory rate from video captured by a hovering unmanned aerial vehicle (UVA). The proposed method estimates heart rate and respiratory rate based on the acquired signals obtained from video-photoplethysmography that are synchronous with cardiorespiratory activity.

Methods: Since the PPG signal is highly affected by the noise variations (illumination variations, subject's motions and camera movement), we have used advanced signal processing techniques, including complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN) and canonical correlation analysis (CCA) to remove noise under these assumptions.

Results: To evaluate the performance and effectiveness of the proposed method, a set of experiments were performed on 15 healthy volunteers in a front-facing position involving motion resulting from both the subject and the UAV under different scenarios and different lighting conditions.

Conclusion: The experimental results demonstrated that the proposed system with and without the magnification process achieves robust and accurate readings and have significant correlations compared to a standard pulse oximeter and Piezo respiratory belt. Also, the squared correlation coefficient, root mean square error, and mean error rate yielded by the proposed method with and without the magnification process were significantly better than the state-of-the-art methodologies, including independent component analysis (ICA) and principal component analysis (PCA).

Keywords: Canonical correlation analysis; Imaging photoplethysmography; Unmanned aerial vehicle; Video magnification technique.

Fig. 1

Experimental setup and data acquisition

Fig. 2

System overview of the proposed

Fig. 3

The iPPG signals for facial ROI (green channel) for a subject in case of a stationary b stationary with 15× magnification, c different face expressions, d talking, and e different illumination conditions

Fig. 4

An example of CEEMDAN decomposition of the iPPG signal in the facial ROI

Fig. 5

The frequency spectrum of decomposed IMFs

Fig. 6

Bland–Altman plots between heart rate measurements obtained by the reference method and heart rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the first scenario

Fig. 7

Bland-Altman plots between heart rate measurements obtained by the reference method and heart rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the second scenario

Fig. 8

Bland-Altman plots between heart rate measurements obtained by the reference method and heart rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the third scenario

Fig. 9

Bland-Altman plots between heart rate measurements obtained by the reference method and heart rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the fourth scenario

Fig. 10

RMSE performance of various heart rate measuring systems for all proposed scenarios

Fig. 11

Bland–Altman plots between respiratory rate measurements obtained by reference method and respiratory rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the first scenario

Fig. 12

Bland–Altman plots between respiratory rate measurements obtained by reference method and respiratory rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the second scenario

Fig. 13

Bland–Altman plots between respiratory rate measurements obtained by reference method and respiratory rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the third scenario

Fig. 14

Bland–Altman plots between respiratory rate measurements obtained by reference method and respiratory rates measured by a the proposed system with magnification, b the proposed system without magnification, c ICA and d PCA for the fourth scenario

Fig. 15

RMSE performance of various respiratory rate measuring systems for all proposed scenarios