Gaze Tracking and Point Estimation Using Low-Cost Head-Mounted Devices

Abstract

In this study, a head-mounted device was developed to track the gaze of the eyes and estimate the gaze point on the user's visual plane. To provide a cost-effective vision tracking solution, this head-mounted device is combined with a sized endoscope camera, infrared light, and mobile phone; the devices are also implemented via 3D printing to reduce costs. Based on the proposed image pre-processing techniques, the system can efficiently extract and estimate the pupil ellipse from the camera module. A 3D eye model was also developed to effectively locate eye gaze points from extracted eye images. In the experimental results, average accuracy, precision, and recall rates of the proposed system can achieve an average of over 97%, which can demonstrate the efficiency of the proposed system. This study can be widely used in the Internet of Things, virtual reality, assistive devices, and human-computer interaction applications.

Keywords: eye tracking; gaze estimation; head-mounted; mobile devices; wearable devices.

Figure 1

Invasive eye tracker [11].

Figure 2

The proposed camera modules.

Figure 3

Three-dimensional diagram of the camera module housing.

Figure 4

Camera module housing.

Figure 5

The head-mounted device.

Figure 6

Comfortable space for the eye in the head-mounted device.

Figure 7

Mobile in the head-mounted device.

Figure 8

System architecture diagram [10].

Figure 9

Image preprocessing flowchart.

Figure 10

Results after setting USB video device class (UVC) parameters.

Figure 11

Use of the checkerboard image to correct the camera.

Figure 12

Results of distortion correction.

Figure 13

Results of bilateral filtering.

Figure 14

Results after using mean shift filtering.

Figure 15

Results of the color balancing.

Figure 16

Image after applying mathematical morphological process.

Figure 17

Binarization image representation using K-means [24,25].

Figure 18

Result using dark group as the threshold.

Figure 19

Result after erosion.

Figure 20

Result of filling the image with flood fill.

Figure 21

Result of subtracting the image before and after filling.

Figure 22

Superimposed mask image.

Figure 23

Image using canny edge detector.

Figure 24

Filling the image in the lower part of the image.

Figure 25

Automatically removed from the background the second time.

Figure 26

Image of connected pupil block.

Figure 27

Flowchart of capturing pupil images [10].

Figure 28

Region of interest (ROI) of the image.

Figure 29

Image Binarization.

Figure 30

Image using the flood-filling method.

Figure 31

Anti-whited low-threshold image.

Figure 32

Multithreshold pupil image.

Figure 33

Flowchart for fitting the pupil ellipse [10].

Figure 34

Pupil image use multithresholds.

Figure 35

Maximum area contour.

Figure 36

Original image.

Figure 37

Reflective highlight contour.

Figure 38

Coincident contour vertex image.

Figure 39

Schematic of optimized pupil vertices.

Figure 40

Pupil elliptical vertex image.

Figure 41

Fitting pupil elliptical image.

Figure 42

The pupil elliptical detection results with the proposed preprocessing techniques.

Figure 43

Pupil elliptical reflection projection [9].

Figure 44

Eyeball model.

Figure 45

Center of the sphere intersects the gaze point vector [9].

Figure 46

Line of sight vector versus screen coordinates.

Figure 47

Flowchart of the system.

Figure 48

Integrated calibration diagram.

Figure 49

Gaze point sequence.

Figure 50

Error angle diagram.

Figure 51

Infrared light image of the eye.

Figure 52

Eyelashes block the pupil.

Figure 53

Too many dark pixels in the eye socket.

Figure 54

Dark pixels of the pupil are too less or unstable shape of ellipse.