A virtual reality paradigm with dynamic scene stimuli for use in memory research

Abstract

Episodic memory may essentially be memory for one's place within a temporally unfolding scene from a first-person perspective. Given this, pervasively used static stimuli may only capture one small part of episodic memory. A promising approach for advancing the study of episodic memory is immersing participants within varying scenes from a first-person perspective. We present a pool of distinct scene stimuli for use in virtual environments and a paradigm that is implementable across varying levels of immersion on multiple virtual reality (VR) platforms and adaptable to studying various aspects of scene and episodic memory. In our task, participants are placed within a series of virtual environments from a first-person perspective and guided through a virtual tour of scenes during a study phase and a test phase. In the test phase, some scenes share a spatial layout with studied scenes; others are completely novel. In three experiments with varying degrees of immersion, we measure scene recall, scene familiarity-detection during recall failure, the subjective experience of déjà vu, the ability to predict the next turn on a tour, the subjective sense of being able to predict the next turn on a tour, and the factors that influence memory search and the inclination to generate candidate recollective information. The level of first-person immersion mattered to multiple facets of episodic memory. The paradigm presents a useful means of advancing mechanistic understanding of how memory operates in realistic dynamic scene environments, including in combination with cognitive neuroscience methods such as functional magnetic resonance imaging and electrophysiology.

Keywords: Deja vu; Episodic memory; Familiarity; Immersion; Role of scenes in episodic memory; Scene recognition; Spatial cognition; Spatial navigation; Unity; Virtual reality.

Fig. 1

Randomization of study-test scenes

Fig. 2

Example of paths traversed within study-test scenes. Note. Participants were guided through scenes in first-person perspective (c, d) along an identical predetermined path (a, b). a Bird’s-eye view of the “Playground” scene displaying the traversal path with a critical right or left turn. b Bird’s-eye view of the corresponding “Junkyard” scene displaying the traversal path with a critical right or left turn. c First-person perspective of the “Playground” scene. d First-person perspective of the “Junkyard” scene

Fig. 3

Familiarity ratings as a function of spatial similarity and experiment (remote, in-person). Note. Average familiarity ratings provided during retrieval failure as a function of Spatial Similarity (Spatially Similar, Spatially Dissimilar) and Experiment (Remote Experiment 2a, In-Person Experiment 2b). Participants across both experiments provided significantly higher familiarity ratings for test scenes corresponding to unrecalled spatially similar study scenes compared to test scenes that did correspond to a studied scene

Fig. 4

Probability of a déjà vu report as a function of spatial similarity and experiment (remote, in-person). Note. Probability of reporting déjà vu during retrieval failure as a function of Spatial Similarity (Spatially Similar, Spatially Dissimilar) and Experiment (Remote Experiment 2a, In-Person Experiment 2b). In both experiments, participants were significantly more likely to report a sense of déjà vu for test scenes that were spatially similar to unrecalled scenes from the study phase than to test scenes that did not correspond to a studied scene

Fig. 5

Familiarity ratings as a function of déjà vu reports and experiment (remote, in-person). Note. Average familiarity ratings provided during retrieval failure as a function of Déjà vu Report (Déjà vu, Non-Déjà vu) and Experiment (Remote Experiment 2a, In-Person Experiment 2b). Across both experiments, participants provided significantly higher familiarity ratings while experiencing déjà vu compared to non-déjà vu

Fig. 6

Feeling-of-prediction ratings as a function of déjà vu reports and experiment (remote, in-person). Note. Average feeling-of-prediction ratings provided during retrieval failure as a function of Déjà vu Report (Déjà vu, Non-Déjà vu) and Experiment (Experiment 2a, Experiment 2b). Overall, regardless of experimental setting, participants provided significantly higher feeling-of-prediction ratings during déjà vu states than non-déjà vu states.

Fig. 7

Probability of commission errors as a function of déjà vu reports and experiment. Note. Probability of reporting making a commission error as a function of Déjà vu Report (Déjà vu, Non-Déjà vu) and Experiment (Remote Experiment 2a, In-Person Experiment 2b). Participants in both experiments were significantly more likely to make a commission error on trials associated with déjà vu reports

Fig. 8

Probability of a feeling-of-prediction as a function of déjà vu reports. Note. Average probability of reporting a feeling-of-prediction during retrieval failure as a function of reported déjà vu state. When participants reported a sense of déjà vu for a test scene, they were significantly more likely to also report having a feeling-of-prediction.

Fig. 9

Probability of a Feeling-of-Prediction as a Function of Spatial Similarity. Note. Note. The average probability of participants reporting a feeling-of-prediction during retrieval failure as a function of spatial similarity. Participants were significantly more likely to report having a feeling-of-prediction during retrieval failure if the test scene spatially corresponded to an unrecalled scene presented at study

Fig. 10

Probability of a commission error as a function of déjà vu reports. Note. Probability of participants making a commission error as a function of reported déjà vu state. When participants were in a reported déjà vu state, they were significantly more likely to make a commission error compared to when they were in a reported non-déjà vu state.

Fig. 11

Effect Size and 95% effect size confidence intervals across Experiments 2a, 2b, and 3. Note. Effect sizes (Cohen’s d) and the corresponding 95% effect size confidence intervals across Experiments 2a, 2b, and 3. Collectively, the effect sizes found in Experiment 3 (VR) tended to be larger than those found in Experiments 2a and 2b (Screen-based)