Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Nov 14;9(11):688.
doi: 10.3390/bioengineering9110688.

A Survey on Physiological Signal-Based Emotion Recognition

Zeeshan Ahmad  1 , Naimul Khan  1
Affiliations

A Survey on Physiological Signal-Based Emotion Recognition

Zeeshan Ahmad et al. Bioengineering (Basel). .

Abstract

Physiological signals are the most reliable form of signals for emotion recognition, as they cannot be controlled deliberately by the subject. Existing review papers on emotion recognition based on physiological signals surveyed only the regular steps involved in the workflow of emotion recognition such as pre-processing, feature extraction, and classification. While these are important steps, such steps are required for any signal processing application. Emotion recognition poses its own set of challenges that are very important to address for a robust system. Thus, to bridge the gap in the existing literature, in this paper, we review the effect of inter-subject data variance on emotion recognition, important data annotation techniques for emotion recognition and their comparison, data pre-processing techniques for each physiological signal, data splitting techniques for improving the generalization of emotion recognition models and different multimodal fusion techniques and their comparison. Finally, we discuss key challenges and future directions in this field.

Keywords: challenges; data annotation; data variability; emotion models; physiological signals; review.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Paper organisation stages.
Figure 2
Figure 2
Plutchik wheel of discrete emotions [39].
Figure 3
Figure 3
2D Valence-arousal Model.
Figure 4
Figure 4
3D Emotion Model.
Figure 5
Figure 5
Steps involving discrete annotation.
Figure 6
Figure 6
Continuous annotation using HCI mechanism [49].
Figure 7
Figure 7
(a) Raw GSR: rest and motion phases. Signals corresponding to the movements involving the right hand are delimited by red lines. (b) GSR decomposition based on EMD, IMF6 and its respective residue [84].
Figure 8
Figure 8
Transformation of ECG signal into GAF, RP and MTF Images.
Figure 9
Figure 9
Error-bars showing inter-subject variability in terms of accuracy across the mean value for 1D WESAD data.
Figure 10
Figure 10
Error-bars showing inter-subject variability in terms of accuracy across the mean value for 2D WESAD data.
Figure 11
Figure 11
Conceptual Visualization of 5-fold cross validation.
Figure 12
Figure 12
Multimodal Multidomain Fusion of ECG Signal.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Phillips L.H., MacLean R.D., Allen R. Age and the understanding of emotions: Neuropsychological and sociocognitive perspectives. J. Gerontol. Ser. B Psychol. Sci. Soc. Sci. 2002;57:P526–P530. doi: 10.1093/geronb/57.6.P526. - DOI - PubMed
    1. Giordano M., Tirelli P., Ciarambino T., Gambardella A., Ferrara N., Signoriello G., Paolisso G., Varricchio M. Screening of Depressive Symptoms in Young–Old Hemodialysis Patients: Relationship between Beck Depression Inventory and 15-Item Geriatric Depression Scale. Nephron Clin. Pract. 2007;106:c187–c192. doi: 10.1159/000104430. - DOI - PubMed
    1. Schluter P.J., Généreux M., Hung K.K., Landaverde E., Law R.P., Mok C.P.Y., Murray V., O’Sullivan T., Qadar Z., Roy M. Patterns of suicide ideation across eight countries in four continents during the COVID-19 pandemic era: Repeated cross-sectional study. JMIR Public Health Surveill. 2022;8:e32140. doi: 10.2196/32140. - DOI - PMC - PubMed
    1. De Nadai S., D’Incà M., Parodi F., Benza M., Trotta A., Zero E., Zero L., Sacile R. Enhancing safety of transport by road by on-line monitoring of driver emotions; Proceedings of the 2016 11th System of Systems Engineering Conference (SoSE); Kongsberg, Norway. 12–16 June 2016; pp. 1–4.
    1. Ertin E., Stohs N., Kumar S., Raij A., Al’Absi M., Shah S. AutoSense: Unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field; Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems; Seattle, WA, USA. 1–4 November 2011; pp. 274–287.

LinkOut - more resources