High-Fidelity Reproduction of Visual Signals by Electrical Stimulation in the Central Primate Retina

Abstract

Electrical stimulation of retinal ganglion cells (RGCs) with electronic implants provides rudimentary artificial vision to people blinded by retinal degeneration. However, current devices stimulate indiscriminately and therefore cannot reproduce the intricate neural code of the retina. Recent work has demonstrated more precise activation of RGCs using focal electrical stimulation with multielectrode arrays in the peripheral macaque retina, but it is unclear how effective this can be in the central retina, which is required for high-resolution vision. This work probes the neural code and effectiveness of focal epiretinal stimulation in the central macaque retina, using large-scale electrical recording and stimulation ex vivo The functional organization, light response properties, and electrical properties of the major RGC types in the central retina were mostly similar to the peripheral retina, with some notable differences in density, kinetics, linearity, spiking statistics, and correlations. The major RGC types could be distinguished by their intrinsic electrical properties. Electrical stimulation targeting parasol cells revealed similar activation thresholds and reduced axon bundle activation in the central retina, but lower stimulation selectivity. Quantitative evaluation of the potential for image reconstruction from electrically evoked parasol cell signals revealed higher overall expected image quality in the central retina. An exploration of inadvertent midget cell activation suggested that it could contribute high spatial frequency noise to the visual signal carried by parasol cells. These results support the possibility of reproducing high-acuity visual signals in the central retina with an epiretinal implant.SIGNIFICANCE STATEMENT Artificial restoration of vision with retinal implants is a major treatment for blindness. However, present-day implants do not provide high-resolution visual perception, in part because they do not reproduce the natural neural code of the retina. Here, we demonstrate the level of visual signal reproduction that is possible with a future implant by examining how accurately responses to electrical stimulation of parasol retinal ganglion cells can convey visual signals. Although the precision of electrical stimulation in the central retina was diminished relative to the peripheral retina, the quality of expected visual signal reconstruction in parasol cells was greater. These findings suggest that visual signals could be restored with high fidelity in the central retina using a future retinal implant.

Keywords: electrical stimulation; epiretinal; ganglion cell; implant; primate; prosthesis; retina.

Figure 1.

Identification of major RGC types using light response and spiking properties in the raphe and peripheral retina. A, Example recording from the raphe (eccentricity, 3.5 mm). Clusters of RF diameter and the first principal component (PC1) of the STA time course (and occasionally the PC1 of the spike train autocorrelation function; data not shown) were used to separate and identify known functionally distinct cell types. The RF mosaics, STA time courses, and spike train autocorrelation functions are shown for each cell type. The ellipses denote the 2σ boundary of a 2D Gaussian fit to the spatial component of the STA. Note: other cell types were also observed, forming different clusters, but for clarity, data only for the five numerically dominant RGC types—ON parasol, OFF parasol, ON midget, OFF midget, and small bistratified—are shown. Black hexagonal outline denotes the approximate location of the electrode array. B, Same as A, but for an example recording in the peripheral retina (8.5 mm temporal equivalent eccentricity).

Figure 2.

LNP model comparison between raphe and peripheral RGCs. A, Example spatial component of the STA (top), STA time course (middle), and contrast–response relationship (bottom) for each of the five numerically dominant cell types in a single raphe preparation. B, Same as A but for a peripheral preparation. C–F, Comparisons of RF diameter (C), RF overlap (D; see Materials and Methods), time of zero crossing of STA time course (E), and nonlinearity index extracted from the contrast–response relation (F; see Materials and Methods) between raphe and peripheral ON and OFF parasol and midget cells. Each data point represents the median value (±1 median absolute deviation) from a single preparation.

Figure 3.

Spiking statistics and correlated firing in ON and OFF parasol cells in the raphe and the periphery. A, Mean (±1 SD) spike train autocorrelation functions for ON (left) and OFF (right) parasol cells in one peripheral and one raphe preparation. B, Probability of spike within 1–25 ms for ON parasol cell autocorrelations (left) and 1–100 ms for OFF parasol cell autocorrelations (see Materials and Methods). Each data point represents the median (+1 median absolute deviation) value in a single preparation. C, Mean (±1 SD) nearest homotypic neighbor spike train cross-correlation functions for ON (left) and OFF (right) parasol cells in one peripheral and one raphe preparation. D, FWHM (see Materials and Methods) of nearest homotypic neighbor cross-correlation functions. Each data point represents the median (±1 median absolute deviation) value in a single preparation. In each box plot, the box denotes the interquartile range, the dashed gray line denotes the median value of all points in the plot, and the bottom and top whiskers denote the first quartile minus 1.5 times the interquartile range and the third quartile plus 1.5 times the interquartile range, respectively.

Figure 4.

Electrical images and distinguishing cell types from intrinsic spiking and electrical features in the raphe. A, Example electrical images of four simultaneously recorded RGCs in a raphe and peripheral recording. The black points denote the locations of individual electrodes on the multielectrode array. The collection of dots of a single color denotes the electrical image of a single cell, with the size of each dot being proportional to the peak recorded voltage on that electrode during the spike of that cell. B, Maximum recorded spike amplitudes within somatic and axonal compartments within the electrical image of each parasol cell. Each data point denotes the median (±1 median absolute deviation) value in a single preparation. C, Spike conduction velocities of parasol cells obtained from axonal electrodes on the electrical image (see Materials and Methods). Each data point denotes the median (±1 median absolute deviation) value in a single preparation. In each box plot, the box denotes the interquartile range, the dashed gray line denotes the median value of all points in the plot, and the bottom and top whiskers denote the first quartile minus 1.5 times the interquartile range and the third quartile plus 1.5 times the interquartile range, respectively. D, Axon spike conduction velocities of ON parasol, OFF parasol, ON midget, and OFF midget cells in a single raphe preparation. For visual clarity, fitted kernel density estimates evaluated on the range of the spike conduction velocity histogram of each cell type are plotted. E, Spike train autocorrelation functions for ON and OFF parasol (left) and ON and OFF midget cells (right) in a single raphe preparation. Projections onto the first two principal components from PCA of the ON and OFF parasol (left) and ON and OFF midget cell (right) autocorrelation functions are shown as insets. Axon spike conduction velocity (D) and spike train autocorrelation functions (E) can together distinguish four of the five numerically dominant cell types. Note that a different raphe preparation was used to show separability using the spike train autocorrelation function. F, Same as D but in the peripheral retina. G. Same as E but in the peripheral retina. A single peripheral preparation was used to show separability with axon spike conduction velocity and spike train autocorrelation function.

Figure 5.

Electrical stimulation in the raphe and peripheral retina. A, Minimum somatic and axonal activation thresholds in ON parasol and OFF parasol cells. Each data point denotes the median (±1 median absolute deviation) value in a single preparation. B, Minimum activation threshold versus bundle activation threshold at each electrode. Each data point denotes the median (±1 median absolute deviation) value in a single preparation. C, ON parasol cell and OFF parasol cell RFs for eight raphe (top two rows) and eight peripheral retina (bottom two rows) preparations. The RF of each cell is colored according to its selectivity index (see Materials and Methods). RFs colored gray were excluded from electrical stimulation analysis because their evoked signals could not be distinguished from noise (see Materials and Methods). D, Average selectivity index values within each cell type and preparation.

Figure 6.

Inference of visual perception in the raphe and peripheral retina with white noise. A, Example linear reconstruction based on evoked RGC responses in individual raphe and peripheral preparations for a single white noise image (pixel size, 110 μm). Each row is a distinct retinal preparation for each retinal location (two preparations per row). The first column shows the original image; the second column shows the optimal reconstruction (i.e., that achievable with perfect control over the firing of each RGC); the third column shows the reconstruction that is achievable by optimized stimulation based on recorded evoked responses (Shah et al., 2019b, ; see Materials and Methods); the fourth column shows the pixels that were incorrectly reconstructed relative to the original image (red, incorrect; blue, correct). Columns 5–7 show the same as columns 2–4, but in the peripheral retina. B, Reconstruction of white noise images with different pixel sizes (352, 220, 176, 110, and 55 μm) in an example raphe and peripheral preparation. Each row is for a distinct white noise image. Columns show the same as in A. C, NMSE (the squared error normalized by L2-norm of target image squared) in the optimal reconstructions (left), the empirical reconstructions (middle), and the fraction of incorrectly reconstructed pixels in the empirical reconstructions (right). Each data point denotes the average of 15 images at each pixel size examined for a single preparation.

Figure 7.

Inference of visual perception in the raphe and peripheral retina with naturalistic images. A, Reconstruction of naturalistic images from the ImageNet database (Fei-Fei et al., 2009) in an example raphe (left) and peripheral (right) preparation after image enhancement with a CAE (Parthasarathy et al., 2017; see Materials and Methods). Each row corresponds to a distinct image. The first column shows the original image; the second column shows the optimal reconstruction (i.e., achievable with perfect control over the firing of each RGC); the third column shows the reconstruction that is achievable by optimized stimulation (Shah et al., 2019b, ; see Materials and Methods). Columns 4 and 5 are the same as columns 2 and 3, but for an example peripheral preparation. Scale bar, 250 μm (visual angle, 1.25°). B, Optimal and empirical NMSE (squared error normalized by L2-norm of target image squared; left) and SSIM (right; Wang et al., 2004) averaged over 100 naturalistic images from the ImageNet database for each raphe and peripheral preparation. Note that in some of the peripheral preparations, the average SSIM for the empirical reconstructions is slightly higher than for the optimal reconstructions, which is possible because the CAE is trained to minimize MSE (see Materials and Methods).

Figure 8.

Estimation of midget cell activation. A, Receptive fields (2σ boundary from a Gaussian fit to the spatial component of the STA) of nearly complete ON parasol and OFF parasol cell populations and partial ON midget and OFF midget cell populations, in a single raphe preparation. B, Receptive fields of simulated complete populations of ON midget and OFF midget cells in the same preparation as A (see Materials and Methods). C, Response probability as a function of distance from stimulating electrode and applied current (see Materials and Methods). This relationship was determined using all of the electrical stimuli and responses from recorded midget cells in A. D, White noise reconstructions from parasol cells and the resulting reconstruction from the activation of simulated midget cells, in a single raphe preparation (A–C; pixel sizes shown, 352, 220, 110, and 55 μm). The first column shows the original image; the second column shows the optimal reconstruction (i.e., achievable with perfect control over the firing of each RGC); the third column shows the empirical reconstruction that is achievable by optimized stimulation based on recorded evoked responses (Shah et al., 2019b, ; see Materials and Methods); the fourth column shows the pixels that were incorrectly reconstructed relative to the original image (red, incorrect; blue, correct); the fifth column shows the reconstruction (third column) summed with the simulated midget cell noise (see Materials and Methods); the sixth column shows the reconstruction summed with the simulated midget cell noise that can be achieved using an early stopping criterion (see Materials and Methods); the seventh column shows the pixels that were incorrectly reconstructed relative to the original image using the early stopping criterion (red, incorrect; blue, correct). E, Naturalistic image reconstructions from parasol cells and the resulting reconstruction from the activation of midget cells in the same preparation as D. The first column shows the original image; the second column shows the optimal reconstruction after application of the same trained CAE as in Figure 7; the third column shows the empirical reconstruction after application of the CAE; the fourth column shows the linear empirical parasol cell reconstruction (linear portion of column 3) summed with the linear simulated midget cell noise; the fifth column shows the resulting image after applying the CAE to the image in column 4. F, Fractional error (see Materials and Methods) between the white noise empirical parasol cell reconstruction (column D3) and the empirical parasol cell reconstruction summed with midget cell noise, after full stimulation (column D5) or after early stopping (column D6). Each data point denotes the average fractional error over 15 images at each pixel size. G, Fractional error between the naturalistic image empirical parasol cell reconstruction and the empirical parasol cell reconstruction summed with the midget cell noise. Fractional error was calculated using both the linear reconstructions as well as the linear reconstructions enhanced by the CAE (see Materials and Methods). The bar plots denote the average fractional error over 100 naturalistic images.