Neurophotonic tools for microscopic measurements and manipulation: status report

Abstract

Neurophotonics was launched in 2014 coinciding with the launch of the BRAIN Initiative focused on development of technologies for advancement of neuroscience. For the last seven years, Neurophotonics' agenda has been well aligned with this focus on neurotechnologies featuring new optical methods and tools applicable to brain studies. While the BRAIN Initiative 2.0 is pivoting towards applications of these novel tools in the quest to understand the brain, this status report reviews an extensive and diverse toolkit of novel methods to explore brain function that have emerged from the BRAIN Initiative and related large-scale efforts for measurement and manipulation of brain structure and function. Here, we focus on neurophotonic tools mostly applicable to animal studies. A companion report, scheduled to appear later this year, will cover diffuse optical imaging methods applicable to noninvasive human studies. For each domain, we outline the current state-of-the-art of the respective technologies, identify the areas where innovation is needed, and provide an outlook for the future directions.

Keywords: blood flow; fluorescence; label free; molecular sensors; multimodal; optical imaging; optogenetics.

Fig. 1

Fluorescent protein engineering.

Fig. 2

Visualization of biomolecules in cleared tissue. (a) Imaging of neurofilament medium unit (NF-M) and GABAB receptor subunit-1 antibodies in cleared, expanded tissue. Scale bars, $50\mu \mathrm{m}$. (b)–(c) Overlay of multi-round immunostained images. Scale bar, $100\mu \mathrm{m}$. Panel (a) is adapted from Ref. ; panels (b) and (c) are adapted from Ref. .

Fig. 3

Super-resolution imaging of fine neuronal structures. (a) Comparison of 2P versus 2P-STED image of dendritic spines imaged in the mouse hippocampus in vivo (adapted from Ref. 36). (b) Correlative STED and SMLM image of dendritic spine morphology and dynamic spatial arrangement of synaptic proteins (blue, neuronal morphology; green, PSD-95 scaffold protein; colored tracks, GluA1 receptor subunit), scale bar 500 nm (adapted from Ref. 37). (c) Super-resolution shadow imaging (SUSHI) of brain tissue, revealing anatomical context of a YFP-labeled neuron (adapted from Ref. 38).

Fig. 4

Expansion microscopy imaging of synaptic proteins. (a) Volumetric image of an inhibitory neuron genetically labelled with EYFP in a $4\times$ expanded mouse brain section stained with antibodies against EYFP (yellow), synaptophysin (magenta) and PSD95 (cyan). (b) Zoom-in view from (a), as indicated by the dashed red box.

Fig. 5

Reconstruction of brain sections treated with MAGIC: (a) mouse, (b) rat, (c) vervet monkey, (d) and human. Scale bar = 1 mm. (e) 3D rendering ($450\times 450\times 60\mu {\mathrm{m}}^3$) of the stack indicated by the red box in d; green and red channels show the myelinated fibers enhanced by MAGIC and the autofluorescence of the cell bodies produced by lipofuscin pigments. Adapted from Costantini et al.

Fig. 6

In vivo 1700-nm optical coherence microscopy (OCM) of the mouse brain through a thinned skull preparation. (a) Minimum intensity projection sagittal images, or “slices,” with different projection thicknesses in the coronal direction, show cortical cytoarchitecture and the corpus callosum (CC), without physical tissue slicing. (b) OCM signal decay (left panel) and averaged coronal image (middle panel) show sub-cortical layers. CC, corpus callosum; Or, stratum oriens; Rad, stratum radiatum; DG, dentate gyrus. Adapted from Zhu et al.

Fig. 7

Ex vivo serial optical coherence scanning (SOCS) of the mouse brain. (a)–(b) Composite images with reflectivity (green) and retardance (red) contrasts for selected regions in mouse brainstem (a) and cerebellum (b). WM, white matter; GL, granular layer; ML, molecular layer; pcf, preculminate fissure. PC indicates a layer of Purkinje cell bodies whose dendrites reside in ML. Scale bar: $320\mu \mathrm{m}$ (length of a square tile).

Fig. 8

Overview of available intensiometric single FP-based GECIs. (a) Representative GFP-based GECIs.^,, (b) Representative mNeonGreen-based GECIs.^, (c) Representative RFP-based GECIs.^,– (d) Representative near infrared FP-based GECIs.^, (e) Qualitative assessment of various GECIs in terms of brightness, sensitivity, localization and in vivo performance.

Fig. 9

Voltage imaging with PeT sensors. (a)–(b) Mechanism of voltage sensing in PeT-based voltage-sensitive fluorophores. (c) Structure of a prototypical voltage-sensitive fluorophore, Berkeley Red Sensor of Transmembrane potential, BeRST 1. (d) Fluorescence image of cultured hippocampal neurons loaded with BeRST 1. Scale bar is $20\mu \mathrm{m}$. (e)–(f) Example traces from simultaneous electrophysiological (black) and optical recording (magenta) of spontaneous voltage fluctuations from neuron 1 (e) and spontaneous activity from neurons 2 to 5 (f); the corresponding neurons are labeled in (d). Adapted from Walker et al. and Liu and Miller, © 2020 American Chemical Society.

Fig. 10

Schematic of chemigenetic probes. A biosensor domain is attached to a self-labeling domain that irreversibly binds a fluorophore ligand. A change in the biosensor domain alters the bound fluorophore properties allowing functional recording of biochemical signals.

Fig. 11

Single cell voltage imaging in awake mice using a wide-field imaging setup. (a) Experimental setup: awake mice were head-fixed under a wide-field microscope. (b) Representative SomArchon-expressing neurons visualized via EGFP fluorescence in cerebral cortex. (c) Voltage imaging time-courses from three representative neurons in hippocampus; horizontal bars denote optogenetic stimulation. Reproduced with permission from Piatkevich et al.

Fig. 12

Genetically encoded sensors for neurotransmitters. (a)–(b) Design scaffolds for sensors based on cognate GPCR (a) or E. coli PBP (b). (c) 2P imaging of dopamine dynamics in the mouse cortex using dLight1.2 when mouse performs a visual-motor learning task. Heat map of dLight1.2 expression pattern in layer 2/3 of M1 cortex is overlaid with computationally defined regions of interest colored by the type of responses observed during the task. (d) Average task-induced dLight1.2 transients (bottom) and mouse running velocity (top) aligned to trial onset (0 s). For details see Ref. .

Fig. 13

Metabolic imaging of the Peredox biosensor of $\mathrm{NADH}/{\mathrm{NAD}}^{+}$ redox, expressed in neurons and astrocytes of acute hippocampal slice. Left: pseudocolor images of the biosensor, which reports high $\mathrm{NADH}/{\mathrm{NAD}}^{+}$ ratio as increased fluorescence lifetime. Right: counterstain with sulforhodamine 101 marks astrocytes specifically and shows that they are the cells with elevated $\mathrm{NADH}/{\mathrm{NAD}}^{+}$ levels in the left-hand images, while neurons have lower levels. Not all cells in the slice express the genetically encoded biosensor, which was introduced with a viral vector. See further discussion of astrocytes in Sec. 3.7.

Fig. 14

Astrocyte subpopulations in the mouse CNS show significant differences in ${\mathrm{Ca}}^{2+}$ signaling. ${\mathrm{Ca}}^{2+}$ transients in sulforhodamine101 (SR101)-labeled astrocytes were detected using Fluo-4. Measurements were made in mouse acute tissue slices containing cortical layer 1 (L1), cortical layers 3-5 (L3-5) and the hippocampal CA1 region (CA1). ${\mathrm{Ca}}^{2+}$ transients (expressed as fluorescence changes relative to baseline: $\mathrm{dF}/{\mathrm{F}}_0$) were initially recorded under conditions of baseline activity (BASE) and after sequential application of tetrodotoxin (TTX) (to isolate astrocytes from neuronal activity) and tetrodotoxin plus the ${\alpha }_1$-adrenergic receptor agonist phenylephrine (PHE). (a) Representative astrocytes (arrowheads) from three brain areas and the ${\mathrm{Ca}}^{2+}$ transients recorded from these cells under the various experimental conditions. Scale bar, $50\mu \mathrm{m}$. (b) Analysis of various transient parameters, recorded under identical experimental conditions. Numerical values are the calculated means for each condition. AUC: area under curve. Modified from Batiuk et al. *p ≤ 0.05, **p ≤ 0.01, ***p ≤ 0.001.

Fig. 15

Structure, optical absorption spectra and basic photophysical properties of probe Oxyphor 2P. In the spectra (right): dotted black line, 1P absorption; solid black lin, 2P absorption; solid dark red line, phosphorescence.

Fig. 16

Bioluminescent tools for imaging, modulating, and integrating. (a)–(c) Luciferases are genetically encoded light emitters that can be employed for imaging neuronal activity when used as a ${\mathrm{Ca}}^{2+}$ dependent split molecule (a), for modulating neuronal activity when tethered to an optogenetic element (b), and for integrating neuronal activity by expressing the light emitter pre- and the light biosensor post-synaptically (c). (d) Example of experiments taking advantage of the bimodal feature of luminopsins: genetically targeted brain circuits were activated chemogenetically (luciferin) during a defined time window of postnatal development; the same neurons were interrogated in adult animals optogenetically (LED).

Fig. 17

Optogenetic tools, genetic targeting and optical techniques. (a) Three major classes of rhodopsin-based optogenetic tools: excitatory pumps and channels (left); inhibitory opsins composed of chloride-conducting channelrhodopsins, proton and chloride pumps, and G-protein-coupled rhodopsins coupled to the inhibitory Gi/o pathway (center); enzyme-rhodopsins (right); shown are the histidine kinase group and the phosphodiesterase group. (b) Expression systems: two examples of gene delivery techniques used to express optogenetic tools in target neuron populations: crossing a recombinase driver mouse line with another that expresses a recombinase-dependent optogenetic actuator (left) or stereotactic injection of a viral vector encoding an optogenetic actuator. (c) Light-delivery techniques for optogenetics: (left) wide-field illumination of neurons expressing an optogenetic actuator through a chronically implanted multimode optical fiber or (right) 2P illumination of individual neurons using spiral-scanning of a diffraction limited spot or parallel illumination of extended shapes using wavefront shaping approaches. Images in panel (b) are from the Allen Institute for Brain Science (https://connectivity.brain-map.org/, experiments #741951571 and #288264301).

Fig. 18

Mapping of brain areas and task-induced neuronal activity with mesoscale ${\mathrm{Ca}}^{2+}$ imaging. (a) Functional mapping of cortical areas. Example maps from three mice (m1, m2, m3) under anesthesia in response to different sensory stimuli: whisker, visual, auditory, forelimb, and hindlimb. Mean evoked activity map is shown for each stimulus type for each mouse. Color denotes normalized fluorescence (ΔF/F). The maps are registered onto the brain atlas (black lines; © 2004 Allen Institute for Brain Science. Allen Mouse Brain Atlas. Available from: http://mouse.brain-map.org/). Overlays of the 5 maps are shown on the right. (b) Parcellation of the dorsal cortical surface into 25 areas. Cortices are color-coded: auditory (green), association (pink), somatosensory + V1 (blue), and motor (red). (c) Behavioral and wide-field imaging setup in mice performing a whisker-dependent memory guided task. (d) Body movement vector in two example hit trials, one active and one passive. (e) Short term memory (STM) maps (averaged during the quiet delay period) for an active and passive trial showing frontal and posterior patches of activity, respectively. Color denotes normalized fluorescence (ΔF/F).

Fig. 19

1P wearable microscopes for cellular-resolution imaging in freely behaving animals. (a) Example 1P miniscope for high-speed, cellular-resolution imaging in freely behaving mice. This implementation is wire-free and weighs 4-5 grams, including a 1.1-gram single-cell lithium-polymer battery. It allows recordings of $\ge 30\min$ duration at 20 Hz, 320 x 320 pixels resolution, x2 pixel subsampling across a $550\mu \mathrm{m}\times 550\mu \mathrm{m}$ FOV. Data is logged onto a MicroSD card for offline analysis. (b) Example raw images of GCaMP6f ${\mathrm{Ca}}^{2+}$ indicator-expressing (top) and computationally identified cells (bottom) in the dorsal hippocampus of control and epileptic mice. Imaging was performed using a tissue-implanted GRIN lens. (c) Spatially tuned activity pattern across 20 example cells from control and epileptic mice trained to run on a 2-m linear track for water reward. Adapted from Ref.  with permission from Nature Publishing Group.

Fig. 20

SCAPE microscopy: elements and application. (a) Top: SCAPE microscopy uses an oblique light sheet to illuminate the sample, and emitted light is collected by the same objective lens. SCAPE sweeps the oblique light sheet back and forth across the sample, de-scanning and rotating returning light to focus onto a stationary camera. Bottom: example images of a drosophila larva, captured as the light sheet scans. (b) Top: experimental imaging configurations to image neuronal activity in zebrafish larvae and awake, behaving mouse. Bottom: visualization of spontaneous neuronal GCaMP activity in the whole brain of a larval zebrafish. Volume rendering shows a maximum intensity projection over time with inset showing time-courses of 6 cells at 6 different depths in the brain. (c) Top and middle: spontaneous activity of apical dendrites of layer 5 neurons in whisker barrel cortex via GCaMP6f imaging in awake behaving mouse. Maximum intensity projection from the top (XY) and side (YZ) with colors denoting time of peak activity. Bottom: time-courses for the seven numbered regions of interest indicated above.

Fig. 21

Deep brain imaging through an optical fiber. (a) Coherent light propagated through a MMF forms a speckle pattern at the output (left). Manipulation of the wavefront entering the MMF yields a desired optical field, e.g., a focal spot (right). (b)–(c) Experimental configuration for in vivo imaging. (d) Inhibitory neurons expressing tdTomato imaged in vivo (panel adapted from Ref. 403). (e) The trade-off between the fiber size and resolution. (f) Dendritic spines of a hippocampal neuron ex vivo resolved using a MMF (bottom). Comparison of the same field of view imaged with a confocal microscope (top). Panels (e) and (f) are adapted from Ref. . (g) In vivo neurons expressing GCaMP6f (left) and their activity visualized as standard deviation over time (right). Bottom traces show the change of signal in time after background (BG) subtraction from two regions of interest (1 and 2). Panel adapted from Ref.  © Optica.

Fig. 22

Large field-of-view, multiregion 2P imaging. (a) FOV available in conventional 2P systems are often confined to $\sim 0.5\mathrm{mm}$, which is smaller than many cortical areas. Larger FOVs (2-mm diameter, or the 5 x 5 mm region of the Diesel2p system provide optical access to multiple cortical areas simultaneously. (b) Neuronal activity can be monitored across these FOVs through rapid beam steering and/or multiplexed beams. (c)–(d) The Diesel2p is a system that provides a large FOV (c) and dual scan engines coupled into a single objective (d) for independent multiplexed imaging and/or 2P optogenetic manipulations of neuronal activity. (e) With large FOV, 2P imaging systems, subcellular resolution is available across a large imaging volume, containing on the order of a million neurons. Adapted from Ref. .

Fig. 23

A summary of state-of-the-art approaches for fast 2P imaging. (a) Commonly used excitation modalities: (i) high NA gaussian beams, (ii) low NA gaussian beams, (iii) Bessel beams, (iv) stereoscopic low NA gaussian beams, (v) line foci for tomographic imaging and (vi) temporally focused, low NA Gaussian beams.^, Top row: simulated intensities of incident beams in the objective back focal plane (BFP) used to generate different excitation PSFs. Middle row: simulated excitation PSFs (red colormap) and the proportional 2P excited fluorescence (green colormap) generated in the focal plane (XY). Bottom row: same as middle but axial (xz) slices. The color bars and principal axes in (i) are applicable to all simulated data apart from (vi) where different colors are used to represent the different wavelengths constituting the ultrafast pulse with central wavelength ${\lambda }_0$ used for temporal focusing. $\tau$ refers to the pulse duration which is shortest at the focal plane where 2P excitation is maximized. Abbreviations: |E| refers to the amplitude of the electric field; ${|\mathrm{E}|}^2$, the intensity, and ${|\mathrm{E}|}^4$, the proportional 2P fluorescence; NA, numerical aperture; TwINS, 2P imaging of neurons using stereoscopy. (b) A summary of different implementations of (i) lateral, (ii) axial, and (iii) temporal multiplexing that have been applied to fast 2P imaging. (i) Left: the incident beam is divided into two beamlets each directed to simultaneously image different regions of interest (ROI). Right: the excitation field is divided into multiple laterally separated spots which sample different portions of the same field of view. (ii) Multiple planes throughout a volume can be imaged simultaneously using axially multiplexed beams or by directing beamlets with different wavelengths to different sample depths. (iii) Since the lifetime of common fluorophores is shorter (shown, not to scale) than the separation between subsequent pulses generated by suitable mode-locked lasers, each pulse can be divided into multiple beamlets, which can be laterally (upper) or axially (lower) displaced to perform fast 2P imaging.^,, (c) Sequential and parallel acquisition modes used for fast 2P imaging. (i)–(v) Methods based on single pixel detection: (i), (ii), (iii), (iv), and (v). (xii)–(x) Common parallel excitation methods combined with camera detection: (vii),^, (viii), and (ix). Abbreviations: PD, point detector; SPAD, single-photon avalanche diode array. (d) (i) Simulated widefield PSF and detection PSFs used for extended depth of field imaging: (ii), (iii), (iv), and (v). Abbreviations: PM, phase mask; WF, widefield; DH, double helix PSF; LF, light field.

Fig. 24

Principles and applications of 3D-AOL microscopy. (a) Schematic of pair of AO deflectors (AODs) with chirped sound waves that curve the optical wavefront of a laser beam (red lines). Modified from Ref. . (b) Two pairs of orthogonally aligned AODs that make up a spherical AOL that can focus and scan a laser beam in 3D. Cubes below illustrates multiple linear (left) and nonlinear (right) drive-based imaging modes. Modified from Ref. . (c) Left: illustration of 3D movement correction with XY and Z scans of a bead interleaved with multiplane imaging. Right: sequence of operation to track a reference object in 3D and compensate imaging for brain movement. Modified from Ref. . (d) Selective imaging of sparsely distributed inhibitory interneurons in the cerebellum. Top left: awake mouse on a treadmill. Top right: locations of somatic ROIs in 3D-AOL imaging volume of mouse expressing GCaMP6f in Golgi cells. Middle: montage of 72 selectively imaged Golgi cell somata. Bottom: activity traces extracted from the cells during locomotion, whisking (WMI) and mild air puff to the whiskers (Puff). Modified from Ref. . (e) Image of pyramidal cell with dendritic branches selectively imaged with multiple line scans (Ribbon scanning). Aligned images of individual dendritic segments with spines clearly visible (middle). Modified from Ref. . (f) Schematic illustrating multiscale imaging functionality of 3D-AOL microscopy, which enables simultaneous imaging of neuronal populations, dendrites, spines and axons present in a local circuit.

Fig. 25

Light Beads Microscopy: elements and application. (a), (b) The Many-fold Axial Multiplexing Module (MAxiMuM) enables 30-fold temporal multiplexing in the axial direction using a cavity-based design resulting in a set of 30 axially focused beams, “Light Beads” at the sample with ${\sigma }_{x,y}\sim 1.1\mu \mathrm{m}$ and ${\sigma }_z\sim 13\mu \mathrm{m}$. A high energy pulse undergoes 30 roundtrips in a slightly imbalanced 8f reimaging cavity and during each round trip a sub-pulse at an appropriate delay is coupled out. Choosing the reflectivity of the output coupler, the exponential falloff of pulse energy can be exactly matched to the required power increase in scattering tissue for all beads. (c) Volumetric recording of 807,748 neurons within $\sim 5.4\times 6\times 0.5{\mathrm{mm}}^3$ at 2.2 Hz within the mouse cortex. 3D rendering of neuron spatial coordinates for a 9-minute recording. (d) Subset of 50 traces from the total of 807,748 extracted neurons, offset: 0.5 $\mathrm{\Delta }\mathrm{F}/{\mathrm{F}}_0$.

Fig. 26

ND-2PE with collinear and displaced beams. (a) Schematic Jablonski energy diagram for D-2PE, D-3PE, and ND-2PE. ${\mathrm{I}}_{\mathrm{f}}$, fluorescence signal intensity; ${\mathrm{I}}_{\mathrm{NIR}}$ and ${\mathrm{I}}_{\mathrm{IR}}$, intensities of the two laser beams; ${\lambda }_{\mathrm{NIR}}$ and ${\lambda }_{\mathrm{IR}}$, the wavelength of the two laser beams; ${\sigma }_{\mathrm{D}2\mathrm{PE}}$, ${\sigma }_{\mathrm{D}3\mathrm{PE}}$, and ${\sigma }_{\mathrm{ND}2\mathrm{PE}}$, cross-sections for D-2PE, D-3PE, and ND-2PE; [C], fluorophore concentration. (b) Simplified schematic of ND-2PE microscope setup: GS-XY, galvanometer scanner; SL, scan lens; TL, tube lens; DM, dichroic mirror; OL, objective lens; PMT, photomultiplier tube. (c) Left: color-coded normalized non-degenerate absorption cross-section for EGFP as a function of the combination of NIR and IR wavelengths. The cross-section value at each combination of wavelengths was normalized by the peak value of this fluorophore. The isocline corresponding to the ground to excited state transition energy is indicated in dashed black line. Right: normalized cross-section values as a function of the equivalent degenerate wavelength $2/{\lambda }_{\mathrm{D}}=1/{\lambda }_{\mathrm{NIR}}+1/{\lambda }_{\mathrm{IR}}$ are shown in red. Independently measured degenerate cross-section values normalized by its peak within the equivalent range of the total photon energy are shown in black. The black dashed line indicates the position of the peak degenerate absorption used for the normalization procedure (930 nm). These wavelengths correspond to the energy isocline shown in panel (b). (d) Simultaneous excitation of four different fluorescent proteins in Brainbow mouse cortical tissue using ND-2PE with ${\lambda }_{\mathrm{NIR}}=850\mathrm{nm}$, and ${\lambda }_{\mathrm{IR}}=1100\mathrm{nm}$. Panel (d) is reproduced with permission from Ref. . (e) Left: D-2PE excites the sample both near the surface and within the focal volume. Right: by selecting the two excitation beams for ND-2PE outside of the excitation spectrum of the fluorophores and spatially displacing the beams, the sample is excited only in the focal volume where two beams overlap. (f) Simulation results for the normalized values of signal, background, and SBR versus the beam displacement from the colinear configuration.

Fig. 27

Deep imaging with 3PM. (a) 3PM of GCaMP6s-labeled neurons in the mouse cortex and the hippocampus. Green, fluorescence; magenta, third-harmonic generation (THG). (b) Activity recording in the Stratum Pyramidale layer of the hippocampus at $\sim 1\mathrm{mm}$ depth. Only a small portion of the 48-minute recording session is shown. Reproduced with permission from Ref. . Springer Nature. (c) Imaging through the intact mouse skull. Red, RFP-labeled neurons in a Brainbow mouse, male, 12 weeks. Green, THG. The zero depth is set just beneath the skull. Reproduced with permission from Ref. . Springer Nature. (d) In vivo 3PM of mouse brain vasculature labeled with quantum dots (Qtracker 655) to $\sim 2.2\mathrm{mm}$ depth. Reproduced with permission from Ref. , © 2019 American Chemical Society.

Fig. 28

2P and 3P wearable microscopes for cellular-resolution imaging in freely behaving animals. (a) Example 2P miniscope for high-resolution imaging at depth. This implementation weighs 4.2 grams and includes an electrically tunable lens (ETL) (top right) for rapid focus adjustment across a $180\mu \mathrm{m}$ focal range (bottom right). Using a microelectromechanical system (MEMS) scanner, the miniscope allows 10 Hz recordings at 512 x 512 pixels resolution across a $420\mu \mathrm{m}$ FOV. (b) Example images of GCaMP6s-expressing neurons acquired at the indicated focal depths (z) in the mouse prefrontal cortex. (c) ${\mathrm{Ca}}^{2+}$ transients from dendritic (top) and cell body regions of interest (ROIs) (bottom) within the focal planes shown in (b). (d) 3P miniscope for deep, high-resolution imaging. This microscope weighs 5 grams and includes a 1.2-m hollow-core photonic bandgap crystal fiber (HC-PBGF) (right) for low-dispersion delivery of the $\sim 1,320\mathrm{nm}$ excitation light-pulses. A plastic optical fiber collects the emitted fluorescence for remote detection by photomultiplier tubes (PMTs). (e) Image of GCaMP6s-labeled rat cortical neurons at the indicated focal depth (z) in the posterior parietal cortex. (f) ${\mathrm{Ca}}^{2+}$ spiking from four example neurons recorded at $1,120\mu \mathrm{m}$ cortical depth. (a)–(f) Adapted with permission from Nature Publishing Group: (a)–(c) from Ref.  and (d)–(f) from Ref. .

Fig. 29

Adaptive optics enables high-resolution multiphoton microscopy at depth. (a) Wavefront distortion of the excitation light can be (b) corrected with a deformable mirror (DM) or liquid crystal spatial light modulator (SLM) to form a diffraction-limited excitation focus. Blue: wavefront. Red: excitation light. Orange: fluorescence. (c) 2P fluorescence images of dendrites and dendritic spines ($606-608\mu \mathrm{m}$ depth) in the mouse cortex. (d) 3P fluorescence images of a neuron ($414\mu \mathrm{m}$ depth) in the mouse spinal cord. (e) AO increases ${\mathrm{Ca}}^{2+}$ transient strength $\mathrm{\Delta }F/F$ and enables detection of orientation selectivity.

Fig. 30

Precision optogenetic modulation of neuronal circuits and behavior. (a) All-optical precision circuit interrogation is enabled by combining 2P imaging and cell-targeted holographic stimulation of a distributed pattern of multiple neurons, typically using a dedicated low-repetition rate (amplified) femtosecond laser. The illumination is extended across each neuron’s membrane by either rapid scanning, reducing the effective numerical aperture (at the cost of optical sectioning) or by parallel illumination of temporally focused extended shapes in two or three dimensions. (b) An array of methods are being developed to calibrate the projected light patterns, adjust as needed, and “smooth” their intensity profile (by minimizing holographic speckle⁵⁴⁷), and in some cases, validate and correct their precise targeting in situ inside the tissue. (c) Beyond target validation, all-optical precision neurophotonic interfaces enable direct readout of the effects of circuit perturbations with simultaneous imaging of neuronal activity (left panel from Ref. 536). This paradigm can be used to infer circuit connectivity and dynamics. One can also infer neuronal activity indirectly through animal behavioral reports of its (perturbed) perceptual experience (right panel, from Ref. 538), which informs us about the behavioral relevance of specific neurons and coding features.

Fig. 31

Angiography and blood flow measurements with OCT. (a) Depth-resolved OCT angiography of the rat somatosensory cortex. Location of OCT cross-sections shown on volumetric angiogram [maximal intensity projection (MIP); superficial vessels in orange-yellow, deep vessels in green]. (b), (c) Cross-sectional angiogram (dynamic intensity, (b)) and static intensity (c) images. (d), (e) Localized changes in dynamic red blood cell (dRBC) content extracted from boxes in (b) and (c), respectively. (f) Depth-resolved measurements show the strongest dRBC responses in middle cortical layers.

Fig. 32

2P imaging of tissue ${\mathrm{pO}}_2$. (a) MIP of a 2P vascular image stack of the top $200\mu \mathrm{m}$ of cerebral cortex. Contrast is due to intravascular fluorescent dye. (b) Zoomed-in measurement plane. (c) Color-coded ${\mathrm{pO}}_2$ measurement points overlaid onto the survey phosphorescence image. (c) ${\mathrm{pO}}_2$ as a function of distance from the center diving arteriole.

Fig. 33

PAT of the brain. (a) In vivo PACT image of a whole rat brain in the coronal plane. (b) In vivo high-resolution PACT image of a mouse brain in the coronal plane. (c) In vivo superresolution PACT image of a mouse brain, where green shows the conventional PACT signals, and magenta shows the enhanced signals from localized droplets. (d) In vivo PAM image of a mouse brain. (e) Cross-sectional PACT image of a mouse brain at 2.8 mm depth.

Fig. 34

Wavefront shaping system design. (a) The wavefront shaping system consists of 5 major steps: Recording, camera data transfer, phase mask computation, phase map transfer, and phase map display and playback. First, the scattered light from a guidestar inside the tissue is captured and collected by the wavefront measurement system. Depending on the type of guidestar, the measurement system may use interferometry to detect the signal from the guidestar (e.g., with an ultrasonic guidestar) or simply measure the bulk signal after filtering (e.g., fluorescence from a particle or cell). The captured data from the camera in the measurement system is then transferred to a computing device such as a PC or FPGA where it is processed to find the desired phase map to send light back to the guidestar. Then, this phase mask is transferred to the SLM and displayed to form the shaped playback wave. (b) An example of a wavefront shaping system used to focus light in brain tissue using time-reversed US-encoded light focusing. The US-tagged light is recorded by the camera and then used to calculate the correct wavefront solution to playback a light focus at the previous location of the ultrasonic focus. (c) The sum of the time required to perform each step of the wavefront sensing and shaping process forms the system latency. Typically, the wavefront measurement time is the most significant portion of the process, but when large pixel number cameras and SLMs are used, the time for data transfer becomes significant as well. Parallel processing on embedded systems such as FPGAs can be used to enhance processing time. Furthermore, SLM technologies such as DMDs can be used to reduce the display time to several tens of microseconds as compared to the much slower, millisecond-order liquid-crystal based SLMs.

Fig. 35

Tapered optical fibers as neuronal interfaces. (a) Example of how modes of different order outcouple at different sections of a TF. Image is in false colors. (b) High resolution patterning allows structuring the non-planar edge of metal-coated tapered optical fibers.

Fig. 36

Multimodal measurements and interrogation of brain activity using transparent electrode arrays and neurophotonic technologies. (a) (Top) Design of a flexible, transparent 16-channel microECoG array for multimodal mapping the dynamics of epileptiform discharges in mouse models. The array is a grid of $50\mu \mathrm{m}\times 50\mu \mathrm{m}$ square electrodes with $500\mu \mathrm{m}$ spacing (total recording area: 1.55 mm × 1.55 mm, total array footprint 2.75 mm × 2.75 mm. Scale bar: 2 mm). (Bottom) Photographs illustrating the optical transparency of the array in the recording area. Scale bars, left to right: 1 mm, $250\mu \mathrm{m}$, and $100\mu \mathrm{m}$. Reproduced from Ref. . (b) Simultaneous optogenetics, recording and hemodynamic imaging in Thy-Cr2 mice through graphene arrays. (Left) Line-scan imaging of arteriolar dilation induced by optogenetic stimulation under two different electrode sites (marker: FITC-dextran, red arrows show line-scan location). Scale bars: $50\mu \mathrm{m}$. (Middle) Average vascular diameter change and (right) light-evoked LFPs recorded at the graphene electrode sites. Reproduced from Ref. . (c) Maps of cortical responses to co-localized electrical stimulation delivered through graphene electrodes in GCamp6f mice. (Top) ${\mathrm{Ca}}^{2+}$ fluorescence intensity in response to cortical electrical stimulation under (left) a graphene and (right) a platinum array of the same size. (Bottom) Spatiotemporal maps of cortical activation following electrical stimulation at varying intensities delivered from a transparent graphene electrode (marked with the red star). Adapted from Ref. . (d) Multimodal analysis of ictal state transitions in acute mouse preparations induced by 4-aminopyridine (4-AP). (Top left) Schematics of the relative graphene contact positions and 4-AP application site. (Top right) Propagation of the ictal wavefront at seizure onset mapped with wide-field ${\mathrm{Ca}}^{2+}$ epifluorescence imaging. (Bottom) ${\mathrm{Ca}}^{2+}$ fluorescence intensity beneath each electrode and simultaneous microECoG recordings from the graphene array during seizure onset. Adapted from Ref. . (e) Closed-loop graphene device for artifact-free optogenetics. (Left) Photographs of the transparent graphene array and (right) the fully assembled battery-powered, closed-loop device integrating the graphene array with an optic fiber for optogenetic stimulation. Adapted from Ref. .

Fig. 37

Concurrent optical imaging and fMRI. (a) Optical imaging implementations within the MRI scanner. Schematic of typical wide-field imaging components (i.). Excitation light is delivered (LED light source), passes through a dichroic mirror, and is focused onto the mouse cortex (1X objective). Emitted fluorescence signal is relayed to a camera and recorded. Example imaging frames from a movie recorded from a mouse expressing GCaMP in glia (Aldh1l1-Cre/Ai162) (top right). Data are collected in-scanner. Blooms of increased activity are in hot colors; epochs of decreased activity are in cool colors. Schematic of a typical fiber photometry experiment (ii.). Excitation light delivered and signal recorded through long fiber optic cables. Both (i.) and (ii.) can be performed within an MRI scanner (iii.) where simultaneous BOLD-fMRI data can be recorded. Wide-field imaging yields measures of cortical activity (2D images), while fiber photometry can target subcortical areas (1D time courses). Simultaneously recorded glia-${\mathrm{Ca}}^{2+}$ (green) and BOLD (orange) spontaneous activity from a cortical roi are plotted (middle). In this example, these signals are moderately correlated (Pearson’s correlation ${\mathrm{R}}^2=0.52$, P < 0.05). (b) In-scanner wide-field imaging. Components and light path are indicated (image reproduced from Ref. 630). For wide-field imaging, optical access to the cortex is gained by resecting the scalp and covering the skull in dental cement, glue, and glass (right). The wide-field imaging data can be registered to the MRI data using the middle cerebral arteries as anatomical landmarks and an MR angiogram. Example multi-modal measurements of spontaneous ${\mathrm{Ca}}^{2+}$ fluorescence and BOLD activity. (c) With multi-modal data registered to a common space (i.), ROIs can be imposed (e.g., Allen Atlas regions) and a connectome computed. Multi-modal spontaneous activity can be used to compute a between contrast transfer function (ii.). Here, gamma-variant fitting was applied to compute a transfer function using simultaneously recorded spontaneous BOLD activity and excitatory neuronal ${\mathrm{Ca}}^{2+}$ activity (Slc17a7/Cam/TeTG6f, N = 6, plotted in blue) or glia ${\mathrm{Ca}}^{2+}$ activity (Aldh1l1-Cre/Ai162, N = 5, plotted in purple). Abbreviations: functional magnetic resonance imaging (fMRI, MRI, MR), light emitting diode (LED), anterior (A), posterior (P), right (R), left (L), dorsal (D), ventral (V), Pearson’s correlation (${\mathrm{R}}^2$), region(s) of interest (roi, rois), signal intensity (SI), arbitrary units (au), blood-oxygen-level-dependent (BOLD), seconds (secs), echo planar imaging (EPI), repetition time (TR), field of view (FOV), radio frequency (RF).

Fig. 38

Computational microscopy synergistically combines optical hardware and computational algorithms.

Fig. 39

Computational microscopy enables novel imaging capabilities. (a) Light field microscopy enables single-shot high-speed volumetric neuronal imaging.^, (b) Miniaturized light field microscope allows volumetric neuronal recording in freely moving mice. (c) Structured illumination microscopy achieves super-resolution reconstruction of dendrites expressing ChR2-GFP on the mouse brain in vivo.

Fig. 40

Deep learning enables state-of-the-art neuronal signal extraction. (a) Vascular segmentation from 2P imaging data. (b) Active neuron segmentation in 2P ${\mathrm{Ca}}^{2+}$ imaging. (c) Denoising 2P ${\mathrm{Ca}}^{2+}$ imaging data.