Air traffic controller work state recognition based on improved xception network

Abstract

In the current context of rapid development of air traffic, the long-time and high-intensity working environment can easily lead to controllers' fatigue state, which in turn affects flight safety. Different from the traditional Mini-Xception pre-training network oriented to the classification task, the study improves it so that it can effectively process multi-dimensional time-series data of air traffic controllers' facial expressions and emotional changes. On its basis, a dynamic time-series data processing module is introduced and combined with a multi-task learning framework and a technique that combines multi-level feature extraction and emotional state analysis to realize the joint recognition of facial expressions and work states, such as fatigue and stress. The experiment findings denotes that the new model has the highest accuracy of 94.36% in detecting eye fatigue, the highest recall rate of 91.68%, and the maximum area under the curve test value of 93.02%. Compared to similar detection models, its average detection time is shortened by 1.9 seconds, with the highest accuracy of 95% in detecting 180 human eye images and an average fatigue detection of 91%. The innovation of the research is to utilize Mini-Xception network for real-time analysis of dynamic features of facial expressions and correlate them with the actual work performance of the controllers, which proposes a new multi-task learning framework, improves the accuracy and stability of the recognition, and provides a new idea and technical support for intelligent monitoring and control of air traffic management system.

Fig 1. Schematic diagram of fatigue level classification.

Fig 2. Schematic diagram of Mini-Xception’s network structure.

Fig 3. Schematic comparison of normal convolution and depth-separable convolution.

Fig 4. Model structure for facial fatigue expression recognition in the face.

Fig 5. Novel human eye fatigue detection model structure.

Fig 6. Network architecture of DCNN and DNN.

Fig 7. Schematic diagram of DCNN-DNN fusion feature processing.

Fig 8. Model performance test results with different parameter settings.

Fig 9. Improved ablation test results for the Mini-Xception network.

Fig 10. Test image display.

Fig 11. Test AP results of different models for 8 graphs.

Fig 12. Average runtime test results for different models.