Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Aug 16:3:956863.
doi: 10.3389/fnrgo.2022.956863. eCollection 2022.

Driver's turning intent recognition model based on brain activation and contextual information

Alexander Trende  1 , Anirudh Unni  2 , Mischa Jablonski  2 , Bianca Biebl  3 , Andreas Lüdtke  1 , Martin Fränzle  4 , Jochem W Rieger  2
Affiliations

Driver's turning intent recognition model based on brain activation and contextual information

Alexander Trende et al. Front Neuroergon. .

Abstract

Traffic situations like turning at intersections are destined for safety-critical situations and accidents. Human errors are one of the main reasons for accidents in these situations. A model that recognizes the driver's turning intent could help to reduce accidents by warning the driver or stopping the vehicle before a dangerous turning maneuver. Most models that aim at predicting the probability of a driver's turning intent use only contextual information, such as gap size or waiting time. The objective of this study is to investigate whether the combination of context information and brain activation measurements enhances the recognition of turning intent. We conducted a driving simulator study while simultaneously measuring brain activation using high-density fNIRS. A neural network model for turning intent recognition was trained on the fNIRS and contextual data. The input variables were analyzed using SHAP (SHapley Additive exPlanations) feature importance analysis to show the positive effect of the inclusion of brain activation data. Both the model's evaluation and the feature importance analysis suggest that the combination of context information and brain activation leads to an improved turning intent recognition. The fNIRS results showed increased brain activation differences during the "turn" decision-making phase before turning execution in parts of the left motor cortices, such as the primary motor cortex (PMC; putative BA 4), premotor area (PMA; putative BA 6), and supplementary motor area (SMA; putative BA 8). Furthermore, we also observed increased activation differences in the left prefrontal areas, potentially in the left middle frontal gyrus (putative BA 9), which has been associated with the control of executive functions, such as decision-making and action planning. We hypothesize that brain activation measurements could be a more direct indicator with potentially high specificity for the turning behavior and thus help to increase the recognition model's performance.

Keywords: automotive; driving simulator; fNIRS; intention classification; machine learning; neuroergonomics.

PubMed Disclaimer

Conflict of interest statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Figures

Figure 1
Figure 1
Virtual reality lab driving simulator with the whole-head fNIRS system during the experiment.
Figure 2
Figure 2
Sketch of intersection. The gap size between two oncoming vehicles is defined as the time that passes after the first oncoming vehicle has crossed the intersection until the second vehicle has crossed the intersection.
Figure 3
Figure 3
Schema of the data separation procedure. The “turn” time interval was extracted 2 s before the actual initiation of the turning maneuver and thus the depressing of the acceleration pedal. The “no turn” phase was extracted 0.5 s before the “turn” phase to minimize the overlap between the two phases.
Figure 4
Figure 4
Accuracies for the three models. The figure shows boxplots for all three models, cross-validation folds, and participants.
Figure 5
Figure 5
Average confusion matrices for all three models.
Figure 6
Figure 6
ROC curves for each of the three models and each of the subjects.
Figure 7
Figure 7
Mean absolute SHAP values for each feature of the combined model. The features are sorted by descending SHAP values.
Figure 8
Figure 8
Beeswarm plot of the feature importance analysis of one subject. Each dot represents one sample of the test dataset. SHAP values for each of these samples are represented by their value on the x-axis. The normalized feature values of each sample are color-coded. Note how the feature values vary systematically in the features with high average SHAP values.
Figure 9
Figure 9
Group-level average Cohen's d brain maps depicting effect sizes computed from channel-wise averaged t-statistics. A tendency for lateralization of activation over the brain is visible.
Figure 10
Figure 10
Beeswarm plots comparison of low and high context feature values for one subject and false negatives of the context only model. Left: Only samples with low context feature values. The combined model learned to utilize the brain activation features for the correct classification. This is indicated by the higher fNIRS SHAP values. Right: For large context feature values, the impact of the context features is on average higher than the brain activation information.
See this image and copyright information in PMC

References

    1. Babiloni C., Ferretti A., Del Gratta C., Carducci F., Vecchio F., Romani G. L., et al. . (2005). Human cortical responses during one-bit delayed-response tasks: an fMRI study. Brain Res. Bull. 65, 383–390. 10.1016/j.brainresbull.2005.01.013 - DOI - PubMed
    1. Buchsbaum M. S., Buchsbaum B. R., Chokron S., Tang C., Wei T. C., Byne W. (2006). Thalamocortical circuits: fMRI assessment of the pulvinar and medial dorsal nucleus in normal volunteers. Neurosci. Lett. 404, 282–287. 10.1016/j.neulet.2006.05.063 - DOI - PubMed
    1. Crozier S., Sirigu A., Lehéricy S., van de Moortele P. F., Pillon B., Grafman J., et al. . (1999). Distinct prefrontal activations in processing sequence at the sentence and script level: an fMRI study. Neuropsychologia 37, 1469–1476. 10.1016/S0028-3932(99)00054-8 - DOI - PubMed
    1. Damm W., Fränzle M., Lüdtke A., Rieger J. W., Trende A., Unni A. (2019). Integrating neurophysiological sensors and driver models for safe and performant automated vehicle control in mixed traffic*, in IEEE Intelligent Vehicles Symposium, Proceedings 2019-June, 82–89. 10.1109/IVS.2019.8814188 - DOI
    1. Dehais F., Dupres A., Di Flumeri G., Verdiere K., Borghini G., Babiloni F., et al. . (2018). Monitoring pilot's cognitive fatigue with engagement features in simulated and actual flight conditions using an hybrid fNIRS-EEG passive BCI, in 2018 IEEE international conference on systems, man, and cybernetics (SMC) (IEEE; ), 544–549.