Fuzzy System-Based Target Selection for a NIR Camera-Based Gaze Tracker

Abstract

Gaze-based interaction (GBI) techniques have been a popular subject of research in the last few decades. Among other applications, GBI can be used by persons with disabilities to perform everyday tasks, as a game interface, and can play a pivotal role in the human computer interface (HCI) field. While gaze tracking systems have shown high accuracy in GBI, detecting a user's gaze for target selection is a challenging problem that needs to be considered while using a gaze detection system. Past research has used the blinking of the eyes for this purpose as well as dwell time-based methods, but these techniques are either inconvenient for the user or requires a long time for target selection. Therefore, in this paper, we propose a method for fuzzy system-based target selection for near-infrared (NIR) camera-based gaze trackers. The results of experiments performed in addition to tests of the usability and on-screen keyboard use of the proposed method show that it is better than previous methods.

Keywords: GBI; NIR camera-based gaze tracker; fuzzy system; gazing at a target to select it.

Figure 1

Overall procedure of the proposed method.

Figure 2

Example of detecting pupil and glint of right eye. (a) Input face image; (b) Search area of right eye; (c) Binarized image; (d) Approximate pupil area detected by CHT, and eye ROI defined on the basis of approximate pupil area; (e) Pupil and glint boundaries detected by Chan–Vese algorithm with adaptive mask.

Figure 3

Flowchart for detecting glint center and pupil region.

Figure 4

Three template graphs for calculating template matching score.

Figure 5

Four images showing the detected centers of glint and the pupil, when a person is looking at the (a) upper-left; (b) upper-right; (c) lower-left; and (d) lower-right calibration positions on a monitor.

Figure 5

Four images showing the detected centers of glint and the pupil, when a person is looking at the (a) upper-left; (b) upper-right; (c) lower-left; and (d) lower-right calibration positions on a monitor.

Figure 6

Relationship between pupil movable region (the quadrangle defined by $\left(P_{x0},P_{y0}\right)$, $\left(P_{x1},P_{y1}\right)$, $\left(P_{x2},P_{y2}\right)$, and $\left(P_{x3},P_{y3}\right)$) in the eye image and the monitor region (the quadrangle defined by $\left(M_{x0},M_{y0}\right)$, $\left(M_{x1},M_{y1}\right)$, $\left(M_{x2},M_{y2}\right)$, and $\left(M_{x3},M_{y3}\right)$).

Figure 7

16 Gabor filters with four scales and four orientations.

Figure 8

ROI for applying Gabor filter based on the user’s gaze position.

Figure 9

Fuzzy method for detecting user’s gaze for target selection.

Figure 10

Designing optimal input fuzzy membership functions for features 1–3 based on maximum entropy criterion. (a) Membership functions for features 1 and 2; (b) Membership functions for features 1 and 3.

Figure 11

Definitions of output membership functions.

Figure 12

Finding the output value of the input membership function for three features: (a) feature 1; (b) feature 2; and (c) feature 3.

Figure 13

Obtaining output score values of fuzzy system using various defuzzification methods. (a) FOM, LOM, and MOM; (b) COG and BOA.

Figure 13

Obtaining output score values of fuzzy system using various defuzzification methods. (a) FOM, LOM, and MOM; (b) COG and BOA.

Figure 14

Experimental setup for proposed method. (a) Example of experimental environment; (b) Three screens of teddy bear (upper), bird (middle), and butterfly (lower) used in our experiments.

Figure 14

Experimental setup for proposed method. (a) Example of experimental environment; (b) Three screens of teddy bear (upper), bird (middle), and butterfly (lower) used in our experiments.

Figure 15

Comparative examples of detection of boundaries and centers of pupil and glint by our method, and the previous method and the ground truth. (a) Detected boundaries of pupil and glint in eye image; (b) Comparison of detection of boundaries of pupil; (c) Comparison of detection of center of pupil; (d) Comparison of detection of boundaries of glint; (e) Comparison of detection of center of glint.

Figure 15

Comparative examples of detection of boundaries and centers of pupil and glint by our method, and the previous method and the ground truth. (a) Detected boundaries of pupil and glint in eye image; (b) Comparison of detection of boundaries of pupil; (c) Comparison of detection of center of pupil; (d) Comparison of detection of boundaries of glint; (e) Comparison of detection of center of glint.

Figure 16

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MIN rule.

Figure 17

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MAX rule.

Figure 18

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MIN rule.

Figure 19

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MAX rule.

Figure 20

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MIN rule.

Figure 21

ROC curves of the classification results of TP and TN data according to different defuzzification methods with the MAX rule.

Figure 22

Comparison of results by proposed method with the results by “Method A” and previous method with experiments involving (a) bear; (b) bird; and (c) butterfly.

Figure 22

Comparison of results by proposed method with the results by “Method A” and previous method with experiments involving (a) bear; (b) bird; and (c) butterfly.

Figure 23

Effect of noise on results by proposed method with the results by “Method A” and previous method with experiments involving (a) bear; (b) bird; and (c) butterfly.

Figure 23

Effect of noise on results by proposed method with the results by “Method A” and previous method with experiments involving (a) bear; (b) bird; and (c) butterfly.

Figure 24

The comparative results of subjective tests of 15 users using the dwell time-based method and the proposed method.

Figure 25

Example of experiment where a user types words on our system.

Figure 26

The comparative results of proposed method with dwell time-based method using virtual keyboard in terms of accuracy and execution time. (a) Accuracy; (b) Average execution time for typing one character; (c) Average execution time for typing one word.

Figure 27

The comparative results of proposed method, with the dwell time-based method using virtual keyboard in terms of convenience and interest.