Feasibility, Enjoyment, and Language Comprehension Impact of a Tablet- and GameFlow-Based Story-Listening Game for Kindergarteners: Methodological and Mixed Methods Study

Abstract

Background: Enjoyment plays a key role in the success and feasibility of serious gaming interventions. Unenjoyable games will not be played, and in the case of serious gaming, learning will not occur. Therefore, a so-called GameFlow model has been developed, which intends to guide (serious) game developers in the process of creating and evaluating enjoyment in digital (serious) games. Regarding language learning, a variety of serious games targeting specific language components exist in the market, albeit often without available assessments of enjoyment or feasibility.

Objective: This study evaluates the enjoyment and feasibility of a tablet-based, serious story-listening game for kindergarteners, developed based on the principles of the GameFlow model. This study also preliminarily explores the possibility of using the game to foster language comprehension.

Methods: Within the framework of a broader preventive reading intervention, 91 kindergarteners aged 5 years with a cognitive risk for dyslexia were asked to play the story game for 12 weeks, 6 days per week, either combined with a tablet-based phonics intervention or control games. The story game involved listening to and rating stories and responding to content-related questions. Game enjoyment was assessed through postintervention questionnaires, a GameFlow-based evaluation, and in-game story rating data. Feasibility was determined based on in-game general question response accuracy (QRA), reflecting the difficulty level, attrition rate, and final game exposure and training duration. Moreover, to investigate whether game enjoyment and difficulty influenced feasibility, final game exposure and training duration were predicted based on the in-game initial story ratings and initial QRA. Possible growth in language comprehension was explored by analyzing in-game QRA as a function of the game phase and baseline language skills.

Results: Eventually, data from 82 participants were analyzed. The questionnaire and in-game data suggested an overall enjoyable game experience. However, the GameFlow-based evaluation implied room for game design improvement. The general QRA confirmed a well-adapted level of difficulty for the target sample. Moreover, despite the overall attrition rate of 39% (32/82), 90% (74/82) of the participants still completed 80% of the game, albeit with a large variation in training days. Higher initial QRA significantly increased game exposure (β=.35; P<.001), and lower initial story ratings significantly slackened the training duration (β=-0.16; P=.003). In-game QRA was positively predicted by game phase (β=1.44; P=.004), baseline listening comprehension (β=1.56; P=.002), and vocabulary (β=.16; P=.01), with larger QRA growth over game phases in children with lower baseline listening comprehension skills (β=-0.08; P=.04).

Conclusions: Generally, the story game seemed enjoyable and feasible. However, the GameFlow model evaluation and predictive relationships imply room for further game design improvements. Furthermore, our results cautiously suggest the potential of the game to foster language comprehension; however, future randomized controlled trials should further elucidate the impact on language comprehension.

Keywords: GameFlow; enjoyment; feasibility; language comprehension; serious gaming.

Figure 1

Main intervention task: (A) story listening (© 2016 Clavis Publishing, Hasselt–Alkmaar, New York; illustrations from Lotta ruimt het op from Diane Put and Rik De Wulf); (B) story rating; (C) multiple-choice questions; (D) reward for each correct response.

Figure 2

Completion of a game session shown in the virtual hub world.

Figure 3

Avatar customization system of the story game.

Figure 4

Overview of the general story appreciation. Each vertical bar represents one story, and the stories are ordered based on their occurrence in the game.

Figure 5

Overview of the general question response accuracy. Each vertical bar represents one question, and the questions are ordered based on their occurrence in the game.

Figure 6

Attrition rate and dropout occurrence throughout the story game. Each vertical bar represents one story, and the stories are ordered based on their occurrence in the game.

Figure 7

Individuals’ game exposure and training duration and their overall distributions. Bold dots represent the final training duration and final game exposure.

Figure 8

Examples of the different gaming profiles of the story game. Complete and compliant player (A). Complete and noncompliant player (B). Incomplete and noncompliant player (C). Dotted lines represent the number of intervention weeks.

Figure 9

Theil–Sen regression outcomes. Predictive relationship between initial mean story appreciation and final game exposure (A). Predictive relationship between initial mean question response accuracy (QRA) and final game exposure (B).

Figure 10

Theil–Sen regression outcomes. Predictive relationship between initial mean story appreciation and final training duration (A). Predictive relationship between initial mean question response accuracy (QRA) and final training duration (B). TD: training duration.

Figure 11

Distribution of the mean question response accuracy (MQRA) of the first game phase.