Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jun 22;22(13):4717.
doi: 10.3390/s22134717.

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Jian Chen  1 , Ming Yan  1 , Feng Zhu  1 , Jing Xu  1 , Hai Li  1 , Xiaoguang Sun  1
Affiliations

Fatigue Driving Detection Method Based on Combination of BP Neural Network and Time Cumulative Effect

Jian Chen et al. Sensors (Basel). .

Abstract

Fatigue driving has always received a lot of attention, but few studies have focused on the fact that human fatigue is a cumulative process over time, and there are no models available to reflect this phenomenon. Furthermore, the problem of incorrect detection due to facial expression is still not well addressed. In this article, a model based on BP neural network and time cumulative effect was proposed to solve these problems. Experimental data were used to carry out this work and validate the proposed method. Firstly, the Adaboost algorithm was applied to detect faces, and the Kalman filter algorithm was used to trace the face movement. Then, a cascade regression tree-based method was used to detect the 68 facial landmarks and an improved method combining key points and image processing was adopted to calculate the eye aspect ratio (EAR). After that, a BP neural network model was developed and trained by selecting three characteristics: the longest period of continuous eye closure, number of yawns, and percentage of eye closure time (PERCLOS), and then the detection results without and with facial expressions were discussed and analyzed. Finally, by introducing the Sigmoid function, a fatigue detection model considering the time accumulation effect was established, and the drivers' fatigue state was identified segment by segment through the recorded video. Compared with the traditional BP neural network model, the detection accuracies of the proposed model without and with facial expressions increased by 3.3% and 8.4%, respectively. The number of incorrect detections in the awake state also decreased obviously. The experimental results show that the proposed model can effectively filter out incorrect detections caused by facial expressions and truly reflect that driver fatigue is a time accumulating process.

Keywords: Adaboost algorithm; BP neural network algorithm; cascade regression tree; fatigue driving detection; time accumulation effect.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Key points selected around the eye for calculating EAR.
Figure 2
Figure 2
Detection results of open and closed eyes using two different methods.
Figure 3
Figure 3
Key points selected around the mouth for calculating MAR.
Figure 4
Figure 4
Schematic diagram of measuring PERCLOS.
Figure 5
Figure 5
Training loss and accuracy of selecting different node numbers in the hidden layer of the BP neural network model. Figures (a–f) represent the training results of the model when the number of hidden nodes is 3–8, respectively.
Figure 5
Figure 5
Training loss and accuracy of selecting different node numbers in the hidden layer of the BP neural network model. Figures (a–f) represent the training results of the model when the number of hidden nodes is 3–8, respectively.
Figure 6
Figure 6
Flow chart of fatigue driving detection based on the BP neural network model.
Figure 7
Figure 7
Flow chart of fatigue driving detection based on the time cumulative effect model.
Figure 8
Figure 8
Detection results of the two methods when the time window is 60 s. (a) shows results based on the BP neural network; (b) shows results based on the time accumulation model.
Figure 9
Figure 9
Detection results of the two methods when the time window is 120 s. (a) shows results based on the BP neural network; (b) shows results based on the time accumulation model.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

References

    1. Sahayadhas A., Sundaraj K., Murugappan M. Detecting Driver Drowsiness Based on Sensors: A Review. Sensors. 2012;12:16937–16953. doi: 10.3390/s121216937. - DOI - PMC - PubMed
    1. Zhao Z., Zhou N., Zhang L., Yan H., Xu Y., Zhang Z. Driver Fatigue Detection Based on Convolutional Neural Networks Using EM-CNN. [(accessed on 20 May 2022)]. Available online: https://www.hindawi.com/journals/cin/2020/7251280/ - PMC - PubMed
    1. Sun W., Zhang X., Peeta S., He X., Li Y., Zhu S. A Self-Adaptive Dynamic Recognition Model for Fatigue Driving Based on Multi-Source Information and Two Levels of Fusion. Sensors. 2015;15:24191–24213. doi: 10.3390/s150924191. - DOI - PMC - PubMed
    1. Chen L., Xin G., Liu Y., Huang J. Driver Fatigue Detection Based on Facial Key Points and LSTM. Secur. Commun. Netw. 2021;2021:5383573. doi: 10.1155/2021/5383573. - DOI
    1. Zhang L., Yang D., Ni H., Yu T. SAE-China 2017: Proceedings of the 19th Asia Pacific Automotive Engineering Conference & SAE-China Congress 2017: Selected Papers. Volume 486. Springer; Singapore: 2017. Driver Fatigue Detection Based on SVM and Steering Wheel Angle Characteristics; pp. 729–738. (Lecture Notes in Electrical Engineering).

LinkOut - more resources