Computational optics for high-throughput imaging of neural activity

Abstract

Optical microscopy offers a noninvasive way to image neural activity in the mouse brain. To simultaneously record neural activity across a large population of neurons, optical systems that have high spatiotemporal resolution and can access a large volume are necessary. The throughput of a system, that is, the number of resolvable spots acquired by the system at a given time, is usually limited by optical hardware. To overcome this limitation, computation optics that designs optical hardware and computer software jointly becomes a new approach that achieves micronscale resolution, millimeter-scale field-of-view, and hundreds of hertz imaging speed at the same time. This review article summarizes recent advances in computational optics for high-throughput imaging of neural activity, highlighting technologies for three-dimensional parallelized excitation and detection. Computational optics can substantially accelerate the study of neural circuits with previously unattainable precision and speed.

Keywords: compressive sensing; computational optics; computer-generated holography; deep learning; neural circuits; non-negative matrix factorization.

Fig. 1

Algorithms for 3D CGH. (a) Superposition algorithms^, generate a phase mask for each focus of the illumination pattern without considering interference. (b) Iterative projection algorithms^– update the phase mask by iteratively constraining the intensity of illumination patterns in the real domain while leaving the phase of illumination patterns unrestricted. The double-constraint GS algorithm restricts both the intensity and the phase of illumination pattern to mitigate speckles. Partial constraint algorithms add error compensation and define unrestricted areas (gray) to improve the performance in the restricted areas (white). (c) Iterative optimization algorithms^, build a differentiable forward model and customize the loss function to optimize the phase mask. The optimization problem can be solved by various gradient descent algorithms. (d) The optimization problem of CGH also can be solved by DNNs that generate high accuracy phase masks at fast speeds.^,

Fig. 2

An example of synthesizing 3D intensity patterns by 4D light field control. Images reproduced with permission from 3D multisite random access photostimulation (3D-MAP). (a) A collimated laser beam illuminates the surface of a DMD with a custom illumination angle set by scanning mirrors. The DMD is synchronized with the scanning mirrors to match the 2D mask of the spatial aperture to the illumination angle. (b) Zoomed in view of the overlapping amplitude masks and illumination angles at the relayed image plane (green) showing how synchronized illumination angles and amplitude masks can generate a focused spot away from the native focal plane (red). Circular patterns labeled by different colors are spatial apertures projected at different times. (c) A focus generated by CGH stimulates the targeted area (blue) in focus but also stimulates non-targeted areas (red) out of focus. 3D-MAP can stimulate only the targeted areas and avoid non-targeted areas by closing the amplitude apertures along propagation directions that project to non-targeted areas (dashed red line). (d) Left: 3D-MAP can simultaneously generate multiple spots in 3D. Right: Experimental measurement of the corresponding 3D fluorescence distribution using a substage camera with a thin fluorescence slide.

Fig. 3

Comparison between conventional sampling and compressive sensing in the spatial domain. (a) Conventional image techniques sample the entire object $x$, so the number of measurements ($P$) equals the number of voxels ($V$). (b) Compressive sensing^– encodes the unknown object $x$ with a known illumination pattern ($A$), and the raw measurement $y$ is a linear combination of multiple voxels of the object. To decompose these voxels of the object, we can solve the inverse problem based on this forward model with the prior of sparsity.

Fig. 4

Space–time domain decomposition can be adopted to various optical microscopy systems for functional brain imaging. (a) The key step of space–time domain decomposition algorithms. The mixed measurement $F$ is decomposed into a product of two low-rank matrices containing spatial components ($S$) and temporal components ($T$), respectively. (b)–(d) Two-photon CGH microscopy uses constrained NMF to demix the calcium activity of individual neurons from dual-plane overlapping images. Scale bar, $50\mu \mathrm{m}$. (e)–(g) Compressive light-field microscopy with ICA and NMF demonstrates functional imaging of 800+ neural structures at a 100 Hz volumetric sampling rate in a live zebrafish. Scale bar, $50\mu \mathrm{m}$. Images reproduced with permission from: (b) Ref.  and (c) Ref. .