EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging study

Abstract

Cognitive human error and recent cognitive taxonomy on human error causes of software defects support the intuitive idea that, for instance, mental overload, attention slips, and working memory overload are important human causes for software bugs. In this paper, we approach the EEG as a reliable surrogate to MRI-based reference of the programmer's cognitive state to be used in situations where heavy imaging techniques are infeasible. The idea is to use EEG biomarkers to validate other less intrusive physiological measures, that can be easily recorded by wearable devices and useful in the assessment of the developer's cognitive state during software development tasks. Herein, our EEG study, with the support of fMRI, presents an extensive and systematic analysis by inspecting metrics and extracting relevant information about the most robust features, best EEG channels and the best hemodynamic time delay in the context of software development tasks. From the EEG-fMRI similarity analysis performed, we found significant correlations between a subset of EEG features and the Insula region of the brain, which has been reported as a region highly related to high cognitive tasks, such as software development tasks. We concluded that despite a clear inter-subject variability of the best EEG features and hemodynamic time delay used, the most robust and predominant EEG features, across all the subjects, are related to the Hjorth parameter Activity and Total Power features, from the EEG channels F4, FC4 and C4, and considering in most of the cases a hemodynamic time delay of 4 seconds used on the hemodynamic response function. These findings should be taken into account in future EEG-fMRI studies in the context of software debugging.

Fig 1. Representative schematics of the acquisition protocol with an example of a run procedure.

The fixed cross is presented on the screen before and after the relevant conditions for analysis, i.e., the text reading condition, neutral code (code comprehension) condition and code inspection condition. The three main conditions order and code snippets examples are randomized in each run.

Fig 2. Representative schematics of the methodology adopted.

The three-stage methodology followed in this work: Preprocessing the EEG data (first block marked as green); Feature Engineering (second block marked as blue); and finally EEG—fMRI Similarity Analysis (third block marked as yellow).

Fig 3. Illustration of the occurrence of the top features obtained in the individual analysis.

Summary of the occurrence of the best 100 features and their corresponding statistical values (mean and corresponding maximum value from the overlap portion metric (d) and also from the absolute correlation values (r) of the significant voxels) from the individual analysis. In (A) is presented the occurrence regarding the feature type; in (B) regarding the EEG channels location; and in (C) the occurrence of the HRF delay used in each subject. In (A) and (B), each colour represents a different subject, and therefore there is a total of 14 different colours.

Fig 4. Illustration of the overlap between EEG-feature correlation map and Insula VOIs, for the individual analysis.

The overlap information presented is over the four different runs from one subject, considering one example of a top selected feature obtained in the individual analysis of the subject (Total Power from the EEG channel FC6). The mean of the absolute correlation values (r) and the overlap portion metric value (d) are also presented for each example. The brain illustrations were generated using the NeuroElf Toolbox v1.1 (developed by Jochen Weber at Columbia University).

Fig 5. Illustration of the occurrence of the top features obtained in the group analysis.

Summary of the occurrence of the best 200 features and corresponding metric values (mean and corresponding maximum value from the overlap portion metric (d) and also from the absolute correlation values (r) of the significant voxels) from the group analysis. In (A) is presented the occurrence regarding the feature type; in (B) regarding the EEG channels location; and in (C) the occurrence of the HRF delay used in each subject.

Fig 6. Illustration of the overlap between EEG-feature correlation map and Insula VOIs, for the group analysis.

The overlap information presented is over the four different runs and considering different subjects, for one example of one of the robust features obtained in the group analysis (Activity from the EEG channel FC4). The mean of the absolute correlation values (r) and the overlap portion metric value (d) are also presented for each example. The brain illustrations were generated using the NeuroElf Toolbox v1.1 (developed by Jochen Weber at Columbia University).