Scaffolding depth cues and perceptual learning in VR to train stereovision: a proof of concept pilot study

Abstract

Stereopsis is a valuable feature of human visual perception, which may be impaired or absent in amblyopia and/or strabismus but can be improved through perceptual learning (PL) and videogames. The development of consumer virtual reality (VR) may provide a useful tool for improving stereovision. We report a proof of concept study, especially useful for strabismic patients and/or those with reduced or null stereoacuity. Our novel VR PL strategy is based on a principled approach which included aligning and balancing the perceptual input to the two eyes, dichoptic tasks, exposure to large disparities, scaffolding depth cues and perception for action. We recruited ten adults with normal vision and ten with binocular impairments. Participants played two novel PL games (DartBoard and Halloween) using a VR-HMD. Each game consisted of three depth cue scaffolding conditions, starting with non-binocular and binocular cues to depth and ending with only binocular disparity. All stereo-anomalous participants improved in the game and most (9/10) showed transfer to clinical and psychophysical stereoacuity tests (mean stereoacuity changed from 569 to 296 arc seconds, P < 0.0001). Stereo-normal participants also showed in-game improvement, which transferred to psychophysical tests (mean stereoacuity changed from 23 to a ceiling value of 20 arc seconds, P = 0.001). We conclude that a VR PL approach based on depth cue scaffolding may provide a useful method for improving stereoacuity, and the in-game performance metrics may provide useful insights into principles for effective treatment of stereo anomalies.This study was registered as a clinical trial on 04/05/2010 with the identifier NCT01115283 at ClinicalTrials.gov.

Figure 1

Stereoacuity transfer after training for clinical and psychophysical tests. (A) Improvement in clinical stereoacuity as a function of initial RCS (filled symbols) and RD3 (open symbols) threshold and comparison with Ding & Levi, 2011 (upside-down triangles) and Vedamurthy et al., 2016 (right-side up triangles). (B) Psychophysical stereoacuity improvement as a function of initial stereo threshold for PDT (triangle), DRS small (small circle), DRS medium (medium circle) and DRS big (big circle). In both figures, colors indicate binocular condition: anisometropia (blue), strabismus (red), stereo-weak (green), and normal stereo (grey). Data under the unity line indicate an improvement in stereoacuity.

Figure 2

Within-block learning example (AS4, strabismic stereo-anomalous participant). From left to right, raw data in block number 3 under Condition 1, 2 and 3. Each asterisk represents depth error (arc seconds) from one trial. The continuous blue line represents a linear fit of the depth error at each trial. The triangle represents the start point and the circle the end point.

Figure 3

Across-block learning example (AS4, strabismic stereo-anomalous participant). From left to right, results under Condition 1, 2 and 3. Results from each block are represented as a vertical line, with a triangle on one end indicating the depth error at the beginning of the block and a circle indicating the depth error at the end of the block. A triangle at the top indicates depth error reduction within a block. Three exponential plots have been superimposed and represent across-block learning. The fits represent an exponential function to the initial error at each block (dotted line), mean error (hashed line), and final error (continuous line).

Figure 4

Across-block learning in four participants. From upper left to bottom right: AA4, stereo-anomalous anisometric; AS4, stereo-anomalous strabismic; N7, stereo-normal; AMS1, stereo-anomalous with micro strabismus. Each graph shows the exponential fit of the end-block depth error in the three conditions: Condition 1, blue dashed line; Condition 2, blue dotted line; Condition 3, dark continuous line. Although N7 performed 60 blocks of training, only first 45 blocks are represented to facilitate comparison.

Figure 5

Box plots of DartBoard in-game performance accuracy, from the exponential fits: Final depth error, PPR, and time constant. Medians and interquartile ranges for each group and condition considered. Depth error values in seconds of arc.

Figure 6

DartBoard in-game performance accuracy initial thresholds and PPR in two cue scaffolding Conditions (1 vs 3) for each group (stereo-normal and stereo-anomalous with strabismic participants plotted separately for less crowding). Each participant is represented as a line, whose start point is a filled circle and end point is an open circle. The start point of the line represents the initial accuracy (arc secs); horizontal line length shows the improvement in game accuracy for Condition 1, and vertical length is the improvement in game accuracy for Condition 3. Points above the diagonal unity line show better performance when all depth cues are present compared to the performance when only retinal disparity is available (as naturally occurs). Lines with angles lower than 45 degrees show greater improvement with all cues than for stereoacuity alone. Stereo-normal participants are represented in gray, stereo-anomalous are represented in different colors depending on subclassification: anisometropic in blue, strabismic in red, stereo-weak in green.

Figure 7

Box plot comparing d’ PPR values between stereo-normal (dark grey) and stereo-anomalous (light grey) groups across stereoacuity demand (400″, 600″, 800″, and 1000″) and Conditions (1, 2, and 3). Condition 1 (left panel), Condition 2 (middle panel), and Condition 3 (right panel). Each symbol represents individual data.

Figure 8

Box plots of Halloween game failures to detect dichoptic targets per 1,000 trials for stereo-normal (grey bars) and stereo-anomalous (white bars) groups for each Condition (1, 2 and 3). Symbols represents data from one participant: stereo-normal (black), anisometropic (blue), strabismic (red) and stereo-weak (green). The horizontal line represents the group median while the whiskers represent the interquartile ranges.

Figure 9

Study and training schematic. Each participant began with a clinical assessment. Followed by clinical and psychophysical stereoacuity tests. Participants then alternated between playing one of two games (Halloween or DartBoard) for 10 h. After every 10 h (20 blocks), clinical and psychophysical stereoacuity tests where administered until 40 h were completed. Lastly, the clinical assessment was administered.

Figure 10

DartBoard and Halloween game screenshots. Top left: Fusion-lock frame calibration for DartBoard (similar in Halloween) to eliminate subjective misalignment angles. Top right: DartBoard 3-AFC suppression task. Bottom left: DartBoard trial example. Bottom right: Halloween trial example.