Exploring the bases for a mixed reality stroke rehabilitation system, part I: a unified approach for representing action, quantitative evaluation, and interactive feedback

Abstract

Background: Although principles based in motor learning, rehabilitation, and human-computer interfaces can guide the design of effective interactive systems for rehabilitation, a unified approach that connects these key principles into an integrated design, and can form a methodology that can be generalized to interactive stroke rehabilitation, is presently unavailable.

Results: This paper integrates phenomenological approaches to interaction and embodied knowledge with rehabilitation practices and theories to achieve the basis for a methodology that can support effective adaptive, interactive rehabilitation. Our resulting methodology provides guidelines for the development of an action representation, quantification of action, and the design of interactive feedback. As Part I of a two-part series, this paper presents key principles of the unified approach. Part II then describes the application of this approach within the implementation of the Adaptive Mixed Reality Rehabilitation (AMRR) system for stroke rehabilitation.

Conclusions: The accompanying principles for composing novel mixed reality environments for stroke rehabilitation can advance the design and implementation of effective mixed reality systems for the clinical setting, and ultimately be adapted for home-based application. They furthermore can be applied to other rehabilitation needs beyond stroke.

Figure 1

Overview of an integrated approach to designing mixed reality rehabilitation systems. An action representation is developed and quantified. Quantification allows for the action representation to be communicated through a media representation to the participant for engagement and intuitive communication of performance to facilitate self-assessment.

Figure 2

Conceptual representation of all elements comprising the action space network. The large central node of the action space network is the action goal. Surrounding nodes contribute to the action goal in varying degrees, and also influence each other within the network. Uniquely shaped nodes represent different contributing parameters of the network.

Figure 3

Simplified conceptual representation of the action space network. The action goal is nested as the central focus of the action, to which other nodes of the network contribute (3a). The visual presentation is simplified by showing only relationships between the goal and two overarching categories of nodes. When focus is on the action goal there is full coupling of categories, shown by full overlap of categories (3b). Decoupling for exploring body/tool function relationships to action is shown by breaking down categories and reducing overlap (3c).

Figure 4

Rehabilitation Approach to action for discriminating behavioral recovery and compensation, adapted from [24] . Three types of action leading to goal accomplishment include activity compensation, activity recovery, and activity recovery with body function/structure recovery.

Figure 5

Action Representation for a Reach and Grasp. Kinematic parameters are listed within seven sub-categories: Four activity level sub-categories (dark background) and three body function level sub-categories (light background). Overlap between categories indicates the general amount of correlation among kinematic parameters with respect to action goal completion. Categories located close to the center of the representation are higher in the hierarchy of training goals, with greater influence on goal completion.

Figure 6

Four feature spaces categorizing feedback for mediated motor learning, provided with example feedback mappings for reaching trajectory and torso compensation from the AMRR system. The example feedback mappings for trajectory and torso compensation are characterized by the location of three unique points placed within each feature space. See descriptions of each feature space in section titled "Feature spaces for designing media feedback".

Figure 7

Three dimensional representation of the sensory modality space during a supported reach to grasp action within the AMRR system. A three dimensional plot illustrates how the coordinates of the feedback experience change over time. The coordinates of the four points represent (1) when the participant is at rest (hand on table) with no audiovisual feedback, (2) when the participant is prompted by audiovisual feedback prior to the reach but still at rest, (3) when the participant is at the mid point of reaching, and finally (4) when the participant grasps the cone.