LIVE-streaming 3D images: A neuroscience approach to full-body illusions

Abstract

Inspired by recent technological advances in the gaming industry, we used capture cards to create and LIVE-stream high quality 3D-images. With this novel technique, we developed a real-life stereoscopic 3D full-body illusion paradigm (3D projection). Unlike previous versions of the full-body illusion that rely upon unwieldy head-mounted displays, this paradigm enables the unobstructed investigation of such illusions with neuroscience methods (e.g., transcranial direct current stimulation, transcranial magnetic stimulation, electroencephalography, and near-infrared spectroscopy) and examination of their neural underpinnings. This paper has three aims: (i) to provide a step-by-step guide on how to implement 3D LIVE-streaming, (ii) to explain how this can be used to create a full-body illusion paradigm; and (iii) to present evidence that documents the effectiveness of our methods (de Boer et al., 2020), including suggestions for potential applications. Particularly significant is the fact that 3D LIVE-streaming is not GPU-intensive and can easily be applied to any device or screen that can display 3D images (e.g., TV, tablet, mobile phone). Therefore, these methods also have potential future clinical and commercial benefits. 3D LIVE-streaming could be used to enhance future clinical observations or educational tools, or potentially guide medical interventions with real-time high-quality 3D images. Alternatively, our methods can be used in future rehabilitation programs to aid recovery from nervous system injury (e.g., spinal cord injury, brain damage, limb loss) or in therapies aimed at alleviating psychosis symptoms. Finally, 3D LIVE-streaming could set a new standard for immersive online gaming as well as augmenting online and mobile experiences (e.g., video chat, social sharing/events).

Keywords: 3D; Augmented reality (AG); Brain stimulation; Full-body illusion (FBI); Game technology; Gaming; Neuroscience methods; Out-of-body experience (OBE); Rehabilitation; Streaming; Virtual reality (VR).

Fig. 1

Stereoscopic 3D full-body illusion combined with HD-tDCS. Front (L) and back (R) view of full-body illusion paradigm based on LIVE-streaming stereoscopic 3D images with Bino 3D Player ( (Lambers, 2012)), here combined with high-definition transcranial direct current stimulation (HD-tDCS). The DC Stimulator Plus (neuroConn) can be seen to the side of the participant fastened on top of a table. Anode (red) vs. cathode (blue) electrode cables run down from the stimulator box up towards participant’s back. Electrodes (1–2 mm thick) are held into position on the scalp with electroconductive paste (Ten20, Weaver). An EEG cap is placed on top to ensure consistent adhesion to the scalp. See Fig. 4 for the electrode montage when the cap is removed

Fig. 2

Stereoscopic 3D camera rig. Phase 1: Design of the stereoscopic 3D camera. Red markings: camera lenses had to be precisely (i) distanced and (ii) aligned in millimeters. Yellow markings: one camera had to be horizontally flipped (see “uncorrected” images). Phase 2: Bino 3D Player-software (1) synchronized, (2) aligned, (3) corrected, (4) prepared, and (5) presented the stereoscopic images in real-time (see “corrected” images, (Lambers, 2012)).

Fig. 3

Procedure full-body illusion * high-definition tDCS. (L) Two cameras captured participants from behind, while they looked at themselves being stroked on the back projected in 3D in front of them. Stroking centered on the back (~20 cm length) at 50 strokes per minute. (R) The brain can be freely stimulated when using plastic 3D goggles to observe the illusion. Cotton cloths are attached to the goggles’ temples to prevent strong spotlights from hitting the lenses

Fig. 4

HD-tDCS montage and P4-P6 electrode positioning. (L) HD-tDCS red “anode” and blue “cathode” concentric center-ring montage to right angular gyrus;. (R) P4-P6 electrode positioning (Brodmann Area 39) in 10-10 International EEG system

Fig. 5

Setup 3D-HDtDCS lab. The participant sat on a stool behind a table located in the middle of the room (~200 cm behind the projection screen). The projector is seen ~400 cm in front of the screen behind the cameras. In front of the stool, a measuring tape is visible that counted out from the participant towards the screen (see right panel). On the measuring tape, participants indicated shifts in self-location or “proprioceptive drift” before and after the illusion (see (de Boer et al., 2020)). The room was kept dark except for two spotlights that illuminated the participant’s back from opposite directions