Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Dec 14;8(8):610.
doi: 10.3390/biomimetics8080610.

Examining the Impact of Digital Human Gaze Expressions on Engagement Induction

Subin Mok  1 , Sung Park  1 , Mincheol Whang  2
Affiliations

Examining the Impact of Digital Human Gaze Expressions on Engagement Induction

Subin Mok et al. Biomimetics (Basel). .

Abstract

With advancements in technology, digital humans are becoming increasingly sophisticated, with their application scope widening to include interactions with real people. However, research on expressions that facilitate natural engagement in interactions between real people and digital humans is scarce. With this study, we aimed to examine the differences in user engagement as measured by subjective evaluations, eye tracking, and electroencephalogram (EEG) responses relative to different gaze expressions in various conversational contexts. Conversational situations were categorized as face-to-face, face-to-video, and digital human interactions, with gaze expressions segmented into eye contact and gaze avoidance. Story stimuli incorporating twelve sentences verified to elicit positive and negative emotional responses were employed in the experiments after validation. A total of 45 participants (31 females and 14 males) underwent stimulation through positive and negative stories while exhibiting eye contact or gaze avoidance under each of the three conversational conditions. Engagement was assessed using subjective evaluation metrics in conjunction with measures of the subjects' gaze and brainwave activity. The findings revealed engagement disparities between the face-to-face and digital-human conversation conditions. Notably, only positive stimuli elicited variations in engagement based on gaze expression across different conversation conditions. Gaze analysis corroborated the engagement differences, aligning with prior research on social sensitivity, but only in response to positive stimuli. This research departs from traditional studies of un-natural interactions with digital humans, focusing instead on interactions with digital humans designed to mimic the appearance of real humans. This study demonstrates the potential for gaze expression to induce engagement, regardless of the human or digital nature of the conversational dyads.

Keywords: ECA; EEG; avatar; digital human; engagement; eye contact; eye tracking; gaze; metaverse; virtual human.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Modal diagram of the investigation.
Figure 2
Figure 2
The process of creating the digital human.
Figure 3
Figure 3
Directions of the digital human’s eyes: (left) gaze avoidance with eyes directed towards the left; (middle) making eye contact; (right) gaze avoidance with eyes directed towards the right.
Figure 4
Figure 4
Experimental procedure.
Figure 5
Figure 5
Setup for the face-to-face conversation.
Figure 6
Figure 6
Setup for both face-to-video and face-to-digital human conversations.
Figure 7
Figure 7
The segmentation of ROIs: left eye, right eye, nose, mouth, forehead, and overall face.
Figure 8
Figure 8
Analysis of subjective evaluation of participants’ responses to positive stimuli.
Figure 9
Figure 9
Analysis of subjective engagement scores between eye contact and gaze avoidance.
Figure 10
Figure 10
Analysis of subjective evaluation of participants’ responses to negative stimuli.
Figure 11
Figure 11
Analysis of brainwave channel FP1 in response to positive stimuli.
Figure 11
Figure 11
Analysis of brainwave channel FP1 in response to positive stimuli.
Figure 12
Figure 12
Analysis of brainwave channel F7 in response to positive stimuli.
Figure 13
Figure 13
Analysis of brainwave channel F7 in response to negative stimuli.
Figure 14
Figure 14
Analysis of brainwave channel T3 in response to negative stimuli.
Figure 14
Figure 14
Analysis of brainwave channel T3 in response to negative stimuli.
Figure 15
Figure 15
Analysis of brainwave channel T4 in response to negative stimuli.
Figure 16
Figure 16
Analysis of gaze to the left eye in response to positive stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Fixation count under the face-to-face condition. (D) Revisit counts in eye-contact condition. (E) Revisit count under the face-to-video condition.
Figure 17
Figure 17
Analysis of gaze to the left eye area in response to positive stimuli. (A) Fixation count under the face-to-face condition. (B) Revisit count under the face-to-video condition. (C) Ratio of ROI under the face-to-video condition. (D) Ratio of ROI under the face-to-digital human condition.
Figure 18
Figure 18
Analysis of gaze to the right eye area in response to positive stimuli. (A) Fixation count under the eye-contact condition. (B) Revisit count under the eye-contact condition. (C) Ratio of ROI under the eye-contact condition.
Figure 19
Figure 19
Analysis of gaze to the right eye area in positive stimuli. (A) Fixation count under the face-to-video condition. (B) Revisit count under the face-to-video condition. (C) Ratio of ROI under the face-to-video condition. (D) Ratio of ROI under the face-to-digital human condition.
Figure 20
Figure 20
Analysis of gaze to the nose area in response to positive stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the eye-contact condition. (D) Revisit count under the gaze-avoidance condition. (E) Ratio of ROI under the eye-contact condition. (F) Ratio of ROI under the gaze-avoidance condition.
Figure 21
Figure 21
Analysis of gaze to the mouth area in response to positive stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the eye-contact condition.
Figure 22
Figure 22
Analysis of gaze to the forehead area in response to positive stimuli. (A) Revisit count under the gaze-avoidance condition. (B) Ratio of ROI under the eye-contact condition.
Figure 23
Figure 23
Analysis of fixation count to the forehead under the face-to-face condition in response to positive stimuli.
Figure 24
Figure 24
Analysis of gaze to the face area in response to positive stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the face-to-face condition. (D) Revisit count under the eye-contact condition.
Figure 25
Figure 25
Analysis of gaze to the left eye area in response to negative stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the eye-contact condition. (D) Ratio of ROI under the eye-contact condition. (E) Ratio of ROI under the gaze-avoidance condition.
Figure 26
Figure 26
Analysis of gaze to the right eye area in response to negative stimuli. (A) Fixation under the eye-contact condition. (B) Revisit count under the eye-contact condition.
Figure 27
Figure 27
Analysis of gaze to the nose area in response to negative stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the eye-contact condition. (D) Revisit count under the gaze-avoidance condition. (E) Ratio of ROI under the eye-contact condition. (F) Ratio of ROI under the gaze-avoidance condition.
Figure 28
Figure 28
Analysis of gaze to the mouth area in response to negative stimuli. (A) Fixation count under the eye-contact condition. (B) Fixation count under the gaze-avoidance condition. (C) Revisit count under the eye-contact condition. (D) Revisit count under the gaze-avoidance condition. (E) Ratio of ROI under the eye-contact condition. (F) Ratio of ROI under the gaze-avoidance condition.
Figure 29
Figure 29
Analysis of gaze to the forehead area in response to negative stimuli. (A) Revisit count under the eye-contact condition. (B) Revisit count under the gaze-avoidance condition. (C) Ratio of ROI under the eye-contact condition. (D) Ratio of ROI under the gaze-avoidance condition.
Figure 30
Figure 30
Analysis of the ratio of ROI to the forehead under the face-to-face condition in response to negative stimuli.
Figure 31
Figure 31
EEG results in response to gaze expression.
See this image and copyright information in PMC

References

    1. Prendinger H., Mori J., Ishizuka M. Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game. Int. J. Hum. Comput. Stud. 2005;62:231–245. doi: 10.1016/j.ijhcs.2004.11.009. - DOI
    1. Zibrek K., Kokkinara E., McDonnell R. The effect of realistic appearance of virtual characters in immersive environments-does the character’s personality play a role? IEEE Trans. Vis. Comput. Graph. 2018;24:1681–1690. doi: 10.1109/TVCG.2018.2794638. - DOI - PubMed
    1. Thompson J., White S., Chapman S. Interactive clinical avatar use in pharmacist preregistration training: Design and review. J. Med. Internet Res. 2020;22:e17146. doi: 10.2196/17146. - DOI - PMC - PubMed
    1. Garety P., Edwards C.J., Ward T., Emsley R., Huckvale M., McCrone P., Rus-Calafell M., Fornells-Ambrojo M., Gumley A., Haddock G. Optimising AVATAR therapy for people who hear distressing voices: Study protocol for the AVATAR2 multi-centre randomised controlled trial. Trials. 2021;22:366. doi: 10.1186/s13063-021-05301-w. - DOI - PMC - PubMed
    1. Creutzfeldt J., Hedman L., Heinrichs L., Youngblood P., Felländer-Tsai L. Cardiopulmonary resuscitation training in high school using avatars in virtual worlds: An international feasibility study. J. Med. Internet Res. 2013;15:e1715. doi: 10.2196/jmir.1715. - DOI - PMC - PubMed

LinkOut - more resources