Soft, wireless periocular wearable electronics for real-time detection of eye vergence in a virtual reality toward mobile eye therapies

Abstract

Recent advancements in electronic packaging and image processing techniques have opened the possibility for optics-based portable eye tracking approaches, but technical and safety hurdles limit safe implementation toward wearable applications. Here, we introduce a fully wearable, wireless soft electronic system that offers a portable, highly sensitive tracking of eye movements (vergence) via the combination of skin-conformal sensors and a virtual reality system. Advancement of material processing and printing technologies based on aerosol jet printing enables reliable manufacturing of skin-like sensors, while the flexible hybrid circuit based on elastomer and chip integration allows comfortable integration with a user's head. Analytical and computational study of a data classification algorithm provides a highly accurate tool for real-time detection and classification of ocular motions. In vivo demonstration with 14 human subjects captures the potential of the wearable electronics as a portable therapy system, whose minimized form factor facilitates seamless interplay with traditional wearable hardware.

Fig. 1. Overview of a soft, wireless periocular electronic system that integrates a VR environment for home-based therapeutics of eye disorders.

(A) A portable, wearable system for vergence detection via skin-like electrodes and soft circuit and relevant therapeutics with a VR system. The zoomed inset details the magnetically integrated battery power source. (B) VR therapy environment that simulates continuous movements of multiple objects in three varying depths of near, intermediate, and distance, which correspond to 1°, 2°, and 3° of eye motions. (C) Zoomed-in view of a skin-like electrode that makes a conformal contact with the nose. (D) Highly flexible, soft electronic circuit mounted on the back of the neck. (E and F) X-ray images of the magnified mesh interconnects (E) and circuit flexion with a small bending radius (F). (G to I) Photos of an ultrathin, mesh-structured electrode, mounted near the eyebrow. (J and K) Two types of eye misalignment due to a disorder: esotropia (inward eye turning) (J) and exotropia (outward deviation) (K), which is detected by the wireless ocular wearable electronics. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.

Fig. 2. AJP parametric study supplemented by the characterization of deposited NP ink.

(A) AJP out of deposition head with a close-up of deposition. (B) Atomization begins in the ultrasonic vial and a carrier gas (N₂) flows the ink droplets through tubing, the diffuser, and the deposition head where a sheath gas focuses the particles into a narrow stream with a diameter of ~5 μm. (C) Platen is heated to ensure evaporation of the solvents, but surface treatment with plasma cleaner enables clean lines (top). Otherwise, the traces tend to form into bubbles over a large area. (D) High focusing ratio with low width/thickness ratio enables lower resistivity. (E) Resistivity measurements from a four-point probe system was determined for various sintering temperatures and time. (F) Ultimately, the resistance of the electrodes needs to be low, so a two-point probe measurement demonstrates the resistance across a fractal electrode in series. (G) AgNPs capped with oleylamine fatty acid that prevents agglomeration end up breaking down after low- and high-temperature sintering. The grain size for small NPs increases at low temperatures, and then, that for bigger NPs increases at higher temperatures. (H) SEM images show the agglomeration and densification of the binder material after sintering. (I) X-ray diffraction analysis shows larger crystallization peaks after sintering and the recrystallization of peaks along (111), (200), and (311) crystal planes. The large peak between (220) and (311) represents Si substrate. (J) Raman spectroscopy demonstrates the loss of carbon and hydrogen peaks from the binder dissociation near wave number 2900 cm⁻¹. (K) Cyclic stretching (50% strain) of the electrode. (L) Cyclic bending (up to 180°) of the electrode.

Fig. 3. Study of ocular vergence and classification accuracy based on sensor positions.

(A) Converging and diverging ocular motions and the corresponding EOG signals with three different targets, placed at 40, 60, and 400 cm away. (B and C) Sensor positioning that resembles the conventional setting (two recording channels and one ground). (D and E) Alternate positions for improved detection of ocular vergence, showing an enhanced accuracy of 87%. (F and G) Finalized sensor positions (ocular vergence 2) that include three channels and one ground, showing the highest accuracy of eye vergence. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.

Fig. 4. Optimization of vergence analysis via signal processing and feature extraction.

(A) Raw eye vergence signals (top) acquired by the wireless ocular wearable electronics in real time and the corresponding derivative signals (bottom). a.u., arbitrary units. (B) Preprocessed data with a band-pass filter and the corresponding derivative, raised to the second power. (C) Further processed data with a 500-point median filter and the corresponding derivative, raised to the sixth power. A coefficient is multiplied to increase the amplitude of the second- and sixth-order differential filters. (D) SNR comparison between the second- and sixth-order differential filters, showing an increased range of the sixth-order data. The sixth-order data are used for thresholding the vergence signals for real-time classification of the dataset. (E) Data from the sliding window are added into the ensemble subspace classifier (shown by the decision boundaries of two dimensions of the feature set).

Fig. 5. Comparison of ocular classification accuracy between VR and physical apparatus.

(A and B) Representative normalized data of eye vergence (red line, average; gray shadow, deviation), recorded with a VR system (A) and a physical apparatus (B). The VR device uses a higher signal gain than the physical setup. (C and D) Normalized peak velocities according to the normalized positions for both convergence and divergence in the VR (C) and the physical setup (D). (E and F) Summarized comparison of averaged classification accuracy (total of six human subjects) based on cross-validation with the VR-equipped soft ocular wearable electronics (91% accuracy) (E) and physical apparatus (80% accuracy) (F).

Fig. 6. VR-enabled vergence therapeutic system.

(A) Example of ocular therapy programs in the VR system: Brock String, which is a string of 100 cm in length with three beads (top images) and measured eye vergence signals (bottom graph). (B) Program, named Eccentric Circle, that uses a set of two cards with concentric circles on each card (top images) and corresponding EOG signals (bottom graph). (C) Continuous use of the VR headset program, showing improved eye vergence, acquired from three human subjects with no strabismus issues from near point convergence. (D) Photos of strabismus exotropia, showing a subject who had difficulty holding the near point convergence during pencil push-ups. (E) Corresponding EOG signals upon convergence and divergence during the pencil push-ups; the left eye shows slower response at convergence, followed by an exotropic incidence after the blink at the near position. Photo credit: Saswat Mishra; photographer institution: Georgia Institute of Technology.