Extended Reality to Assess Short-Term Spatial Memory-A Comparative Study of Mixed Reality, Augmented Reality, and Virtual Reality

Abstract

A Mixed Reality (MR) application using an optical see-through headset was developed to assess short-term spatial memory. A study with 29 participants was conducted. Data from this study were compared to two previous studies using mobile Augmented Reality (AR) and Virtual Reality (VR) with headsets. When comparing the three technologies (MR, AR, VR) for the performance variables, there were no statistically significant differences for either the total number of correctly placed objects or the total number of attempts. However, the MR application required more time than the AR and VR applications in the evaluation phase and more time than the VR application in the learning phase. Our arguments for the longer time are the novelty of the MR application for the participants and the characteristics of the applications. The key results from the MR study include the following: (1) the objects used in the MR application were correctly positioned on a map, which implies that the memory acquired with the MR application is effectively transferred to the user's mental map; (2) for the performance variables, there were no significant differences in the results by gender; (3) and the usability rating decreased with computer experience. The results show that the MR application is effective for spatial memory assessment and was well rated by the participants. The three technologies, along with suitable hardware, are effective for spatial memory assessment. However, MR using optical see-through headsets offers advantages over mobile AR and VR using headsets, discussed in this publication.

Keywords: HoloLens 2; assessment; augmented reality; mixed reality; optical see through; short-term memory; spatial memory; user experience; user performance; virtual reality.

Figure 1

The object selection menu for setting up a task.

Figure 2

Selected 3D objects for the MR application: (A) a violin; (B) a set of books; (C) a mug; (D) an earth globe; (E) a frog-shaped decorative object; (F) a toy car; (G) a camera; and (H) a mushroom-shaped decorative object.

Figure 3

Room layout and object positions used in the MR study. The objects are highlighted with red-bordered squares.

Figure 4

Virtual objects in the physical environment as seen by the participant.

Figure 5

Box plots comparing the performance variables for the MR (red), AR (green), and VR (blue) applications.

Figure 6

Interaction plots for the performance variables when using the MR application and taking into account the age and gender of the participants. The triangles represent the women’s group, and the red rhombuses represent the men’s group. Black solid lines connect women, while red dashed lines connect men. Lines connect only consecutive points within the same group to highlight trends and avoid overlaps.

Figure 7

Radial graph showing the mean of the subjective variables considering the MR, AR, and VR applications.

Figure 8

Scatter plots for the positive correlations between satisfaction and enjoyment and presence and enjoyment. The green area represents a 95% confidence level interval for predictions from a linear model. The red line represents the fitted regression line, showing the linear relationship between variables.

Figure 9

Scatter plots for the negative correlation between concentration and computer experience (left) and the negative correlation between usability and computer experience (right). The green area represents a 95% confidence level interval for predictions from a linear model. The red line represents the fitted regression line, showing the linear relationship between variables.