The user-centered design as novel perspective for evaluating the usability of BCI-controlled applications

Abstract

Albeit research on brain-computer interfaces (BCI) for controlling applications has expanded tremendously, we still face a translational gap when bringing BCI to end-users. To bridge this gap, we adapted the user-centered design (UCD) to BCI research and development which implies a shift from focusing on single aspects, such as accuracy and information transfer rate (ITR), to a more holistic user experience. The UCD implements an iterative process between end-users and developers based on a valid evaluation procedure. Within the UCD framework usability of a device can be defined with regard to its effectiveness, efficiency, and satisfaction. We operationalized these aspects to evaluate BCI-controlled applications. Effectiveness was regarded equivalent to accuracy of selections and efficiency to the amount of information transferred per time unit and the effort invested (workload). Satisfaction was assessed with questionnaires and visual-analogue scales. These metrics have been successfully applied to several BCI-controlled applications for communication and entertainment, which were evaluated by end-users with severe motor impairment. Results of four studies, involving a total of N = 19 end-users revealed: effectiveness was moderate to high; efficiency in terms of ITR was low to high and workload low to medium; depending on the match between user and technology, and type of application satisfaction was moderate to high. The here suggested evaluation metrics within the framework of the UCD proved to be an applicable and informative approach to evaluate BCI controlled applications, and end-users with severe impairment and in the locked-in state were able to participate in this process.

Figure 1. Transfer of the matrix based speller paradigm to the Qualilife software.

Left: To adapt end-users to the flashing of dots, those were placed in each cell of the well familiar matrix. Instead of the letters those dots were flashed. Right: Screen shot of the Qualilife communication application. The now familiar red dots were assigned to each option of the Qualilife communication and control surface. Red dots appear randomly at each possible “button” to press. Attention needs to be focused on the specific button to be pressed by counting how often the red dot is appearing.

Figure 2. “Grau-Gelb” (engl. “grey-yellow”) - a painting created with the Brain Painting application by a locked-in end user with amyotrophic lateral sclerosis (© J Thiele, with permission).

Figure 3. Ratings for the BCI specific items (explanation see text).

Figure 4. Key user-centered design activities (from [12]) adapted to BCI-controlled applications.

If the application matches the individual end-user's needs, it will very likely be used in daily life.