Mental Fatigue Monitoring Using a Wearable Transparent Eye Detection System

Abstract

We propose mental fatigue measurement using a wearable eye detection system. The system is capable of acquiring movement of the pupil and blinking from the light reflected from the eye. The reflection is detected by dye-sensitized photovoltaic cells. Since these cells are patterned onto the eyeglass and do not require external input power, the system is notable for its lightweight and low power consumption and can be combined with other wearable devices, such as a head mounted display. We performed experiments to correlate information obtained by the eye detection system with the mental fatigue of the user. Since it is quite difficult to evaluate mental fatigue objectively and quantitatively, we assumed that the National Aeronautics and Space Administration Task Load Index (NASA-TLX) had a strong correlation with te mental fatigue. While a subject was requested to conduct calculation tasks, the eye detection system collected his/her information that included position, velocity and total movement of the eye, and amount and frequency of blinking. Multiple regression analyses revealed the correlation between NASA-TLX and the information obtained for 3 out of 5 subjects.

Keywords: dye sensitized photovoltaic device; eye; fatigue; mental state; micro; microelectromechanical systems; monitoring; sensor; wearable.

Figure 1

(a) See-through-type wearable eye-tracking system. The colored parts are dye-sensitized photovoltaic cells that detect the light reflected from the eye and deduce the pupil position and eye blink. A charge-coupled device (CCD) camera is used to detect the line of sight of the user. When we only need to detect the movement of the eye and eye blinking, the CCD camera is not used and no external power is necessary. (b) The eye-tracking system in use. Since the distance between the sensor cells and the eye is a few centimeters, the user does not notice the cells and can see through them as if he/she were wearing sunglasses with the color of the dye.

Figure 2

(a) Four photovoltaic cells patterned onto the eyeglasses. (b) The horizontal position of the pupil can be deduced from the difference between V_L and V_R, which are the average of the two left cells (V_ul and V_dl) and the two right cells (V_ur and V_dr). (c) The vertical position of the pupil can be deduced from the difference between V_U and V_D, which are the averages of the two upper cells (V_ul and V_ur) and the two bottom cells (V_dl and V_dr).

Figure 3

(a) Output voltage and (b) the first derivative when the subject blinks. The system clearly detects the eye blinks. The red line in Figure 3b represents the threshold value for blink detection.

Figure 4

Experimental setup. The subject wears the eye tracker. A number with 3 or 4 digits is shown by the projector on the white board located 1 m in front of the subject. The switch to change the question is placed on the desk.