Demixing fluorescence time traces transmitted by multimode fibers

Abstract

Optical methods based on thin multimode fibers (MMFs) are promising tools for measuring neuronal activity in deep brain regions of freely moving mice thanks to their small diameter. However, current methods are limited: while fiber photometry provides only ensemble activity, imaging techniques using of long multimode fibers are very sensitive to bending and have not been applied to unrestrained rodents yet. Here, we demonstrate the fundamentals of a new approach using a short MMF coupled to a miniscope. In proof-of-principle in vitro experiments, we disentangled spatio-temporal fluorescence signals from multiple fluorescent sources transmitted by a thin (200 µm) and short (8 mm) MMF, using a general unconstrained non-negative matrix factorization algorithm directly on the raw video data. Furthermore, we show that low-cost open-source miniscopes have sufficient sensitivity to image the same fluorescence patterns seen in our proof-of-principle experiment, suggesting a new avenue for novel minimally invasive deep brain studies using multimode fibers in freely behaving mice.

Fig. 1. Concept of single-source resolved fiber photometry (demixed fiber photometry).

From left to right: ground-truth excitation mimicking neuronal activity is performed by using a DMD, which can selectively excite a set of fluorescent emitters on the sample with a given time trace, likewise in^,. A short (8 mm long) multimode fiber typically implanted in optogenetics or fiber photometry experiments (NA = 0.39, 200 µm core diameter, step-index fiber) is placed almost touching the sample (distance of ≈ 50 µm) to collect the fluorescence dynamics of each source. Due to its proximity to the sample, the fiber’s effective FoV is expected to be slightly larger than the core size. Fluorescence light inside the multimode fiber is subject to multimodal mixing during propagation, which scrambles/mixes the emitters’ wavefront similarly to any scattering media. The transmitted superimposed signal (#1, #2, and #3) consists of fluorescence transient patterns, i.e., 2D patterns (w1, w2, w3) that fluctuate in intensity over time with typical calcium transient profiles (h1, h2, h3)^,. A video is recorded with a camera and a post-processing step using a spatio-temporal demixing algorithm (unconstrained NMF) is applied to disentangle the overlapped transient patterns into individual 2D spatial fingerprints and their corresponding singular time trace profiles that should match the GT excitations. The optical setup and raw data videos details are fully described in Fig. S1.

Fig. 2. Results of a proof-of-principle experiment performed with 6 fluorescent beads.

From (a) to (d) we have: (a) the ensemble temporal activity (fiber photometry), (b) the superimposed pattern image of 6 fluorescent bead fingerprints when simultaneously excited (imaged detected via sCMOS camera) (see “Methods” and Fig. S1); (c) the short MMF located at a distance of 60 ± 10 µm from the fluorescent beads; (d) a CMOS Basler camera with the ground truth image of the sample (see Fig. S1 for setup scheme details). e The ground truth (GT) fingerprint patterns obtained from each bead when they were individually excited. f The fingerprint patterns obtained via NMF demixing are to be compared with the GT patterns in (e). g The individual temporal activity traces of the sources obtained with NMF (blue) and their corresponding GT traces (gray). The NMF trace (#6) was not recovered well by NMF since bead #6 was localized very close to the fiber core edge, therefore yielding low signal/contrast of its pattern (see GT scattering fingerprint of bead #6 in (e)). h The GT-NMF time trace correlations. The average diagonal value of the first 5 beads was <δ_g,n > = δ_avg = 85.4% with ${\sigma }_{\delta }$ = 3.6%. To better evaluate the off-diagonal elements (time trace cross-talk), we subtract them from their corresponding GT-GT coefficients. Then, we averaged the absolute values of these differences and we obtained the mean absolute error of ${\zeta }_{avg}=$ 7.06% with a standard deviation of${\sigma }_{\zeta }=$ 7.29% for the first 5 beads (see Supplementary Note 2). i The GT-GT temporal trace correlation table. Importantly, the GT-GT correlation coefficients show that although each GT trace was unique over time (singular profile), GTs from different sources were not fully uncorrelated. For example, GT traces of beads #3 and #4 were fairly correlated (γ_3,4 = γ_4,3 = 31.2%, in (i)) and had a very clear spatial overlap (see GT and NMF scattering fingerprints #3 and #4 in (e) and (f)).

Fig. 3. Results of a proof-of-principle experiment performed with 26 fluorescent beads including a neuropil background source.

The bead #20 is modeling the neuropil (highlighted in orange). From (a) to (d) we have: (a) the photometric (ensemble) time trace, which is the sum of 26 time traces; (b) the sCMOS detected image of the spatially overlapped fingerprint patterns from 26 fluorescent beads probed by the short MMF (see “Methods”); (c) the short MMF located at a distance of 60 ± 10 µm from the sample; (d) the ground truth image of the sample (backpropagated fluorescence image detected from a CMOS Basler camera, see details of the setup in Fig. S1). e The ground truth (GT) fingerprint patterns obtained from each bead when they were individually excited. f The fingerprint patterns obtained via NMF are to be compared with the GT patterns in (e). g Top: the individual temporal activity traces obtained with NMF (blue) and their corresponding GT traces (gray). Bottom: the photometric ensemble signal from the recorded video (black line), which is the sum of all individual traces. The fluorescence intensity in all traces in the figure are normalized to 1. h The GT-NMF time trace correlations. The average diagonal value of the first 22 beads was <δ_g,n > = δ_avg = 86.0% with ${\sigma }_{\delta }$ = 5.4%. To better evaluate the off-diagonal elements (time trace cross-talk), we subtract them from their corresponding GT-GT coefficients. Then, we averaged the absolute values of these differences and we obtained the mean cross-talk of ${\zeta }_{avg}=$ 4.4% with a standard deviation of${\sigma }_{\zeta }=$ 3.7% for the first 22 beads (see Supplementary Note 2). i The GT-GT temporal trace correlation table showing that the ground truth traces were not orthogonal.

Fig. 4. Results of a proof-of-principle experiment performed with 26 fluorescent beads behind a Parafilm M® layer.

The bead #13 is the source mimicking neuropil background signal (highlighted in orange). From (a) to (d) we have: (a) the photometric (ensemble) time trace, which is the sum of 26 time traces; (b) the sCMOS detected image of the spatially overlapped fingerprint patterns from 26 fluorescent beads simultaneously probed by the short MMF (see “Methods”); (c) the short MMF located at a distance of 60 ± 10 µm from the sample; (d) the ground truth image of the sample (backpropagated fluorescence image detected from a CMOS Basler camera, see setup in Fig. S1). e The ground truth (GT) fingerprint patterns obtained from each bead when they were individually excited. f The fingerprint patterns obtained via NMF are to be compared with the GT patterns in (e). g Top: the individual temporal activity traces obtained with NMF (blue) and their corresponding GT traces (gray). Bottom: the photometric signal from the recorded video (black line), which is the sum of all individual traces. The fluorescence intensity in all traces in the figure are normalized to 1. h The GT-NMF time trace correlations. The average diagonal value of the first 20 beads was <δ_g,n > = δ_avg = 77.0% with ${\sigma }_{\delta }$ = 11.9%. To better evaluate the off-diagonal elements (time trace cross-talk), we subtract them from their corresponding GT-GT coefficients. Then, we averaged the absolute values of these differences and we obtained the mean cross-talk of ${\zeta }_{avg}=$ 5.9% with a standard deviation of${\sigma }_{\zeta }=$ 5.6% for the first 20 beads (see Supplementary Note 2). i The GT-GT temporal trace correlation table showing that the ground truth traces were not orthogonal - some of them were correlated.

Fig. 5. Results of a proof-of-principle experiment performed in conditions simulating dominant neuropil activity.

The sample consists of 21 fluorescent beads, where 10 beads had sparse and unique time traces and the other 11 had the same non-sparse neuropil-like time trace (dynamic background). The beads with “neuropil-like” background activity are highlighted in orange (#1, #11, #12, #13, #14, #15, #16, #17, #18, #19, and #20). The remaining beads (#2, #3, #4, #5, #6, #7, #8, #9, #10, and #21) had unique sparse neuronal activity time traces mimicking signal from neuronal cell bodies (target sources). a The photometric (ensemble) time trace, which is the sum of all the 21 time traces; (b) the sCMOS detected image of the spatially overlapped fingerprint patterns from 21 fluorescent beads simultaneously probed by the short MMF (see “Methods”); (c) the short MMF located at a distance of 60 ± 10 µm from the sample; (d) the ground truth image of the sample (backpropagated fluorescence image detected from a CMOS Basler camera, see setup in Fig. S1). e The ground truth (GT) fingerprint patterns obtained from each bead when they were individually excited. f The fingerprint patterns obtained via NMF. Note that the NMF pattern #1 is the neuropil pattern due to the spatial overlap of 11 sources (highlighted with orange squared boxes). g Top: the individual time traces obtained with NMF (blue) and their corresponding GT traces (gray). Bottom: the photometric signal from the recorded video (black line), which is the sum of all individual traces. The fluorescence intensity in all traces in the figure are normalized to 1. h The GT-NMF time trace correlations. The average diagonal value of the first 10 beads was <δ_g,n > = δ_avg = 86.0% with ${\sigma }_{\delta }$ = 5.5%. To better evaluate the off-diagonal elements (time trace cross-talk), we subtract them from their corresponding GT-GT coefficients. Then, we averaged the absolute values of these differences and we obtained the mean cross-talk of ${\zeta }_{avg}=$ 6.1% with a standard deviation of${\sigma }_{\zeta }=$ 5.7% for the first 10 beads (see Supplementary Note 2). i The GT-GT temporal trace correlation table showing that the ground truth traces were not orthogonal.

Fig. 6. Validation of the concept of single-activity resolved fiber photometry with short multimode fibers in a brain tissue environment (in vitro).

Sample: Gad-EGFP neurons fixed in a 50 µm brain slice, sealed in between 2 coverslips to keep the humidity of the tissue (see “Methods”). a The ensemble photometric time trace of this experiment. b The fiber proximal end image of 4 neurons’ fingerprint patterns spatially overlapped on the sCMOS camera chip. c An illustration of the short MMF placed above the top coverslip of the sample, at a distance of ≈ 60 ± 10 µm from it; and (d) the GT image of the sample highlighting the 4 selected neurons to be excited (structurally labeled). e The GT fingerprint patterns are obtained from each neuron when individually excited. f The fingerprint patterns retrieved via NMF are in good agreement with the GT patterns in (e). g The demixed temporal activity traces are sorted in descending GT-NMF correlation order (from the most correlated time traces on top to the least correlated time traces on the bottom). Traces in blue are retrieved by NMF and temporal traces in gray are their GT. h The GT-NMF temporal trace correlation coefficients. i The GT-GT temporal correlations. The average diagonal value in (h) of the 4 neurons was <δ_g,n > = δ_avg = 86.7%, with standard deviation of = 2.8%. Regarding the non-diagonal elements (cross-talk), the mean absolute error taking into account the GT-GT coefficients was ${\zeta }_{avg}=$ 8,95% with a standard deviation of${\sigma }_{\zeta }=$ 8.02% (see Supplementary Note 2). Again, although each GT trace was unique in time (singular), they were not fully uncorrelated as we can see in the GT-GT correlation traces (i). Interestingly, neurons #1 and #2 (i.e., the two best NMF retrieved results) were also the most temporally correlated ones in the GT excitation (γ_1,2 = γ_2,1 = 25.0%).

Fig. 7. Novel microendoscopy concept using a short MMF and a miniscope: the MiniDART.

a A typical fingerprint pattern from a single-fluorescent bead (10 µm diameter) probed by using only the miniscope excitation and miniscope detection through the multimode fiber. b The experimental setup to probe scattering patterns from the short MMF includes (i) the miniscope, (ii) a customized titanium base plate (YMETRY®) to hold the miniscope, (iii) a ferrule (Thorlabs SFLC230-10) that rigidly holds the multimode fiber (iv) within it, (v) a sample consisting of a single fluorescent 10 µm bead (spatial density <1 bead/cm²), and (vi) a customized titanium tweezer (YMETRY®) to hold the ferrule. c Scattering fingerprint patterns at the proximal end (bottom row) depending on the radial position (d) of the single-bead at the distal end (bottom row). Position (d) (red arrow) is indicated in relation to the fiber axis (the bead is represented as a blue spot in the zoom of (b) and the top row images of (c), while the axial center of the fiber is represented as a fixed black dot in the top row images of (c)). Each pattern acquisition in the proximal end (bottom row of (c)) corresponds to 10 µm steps of the bead from the fiber central axis in the distal end (top row of (c)). The bigger the distance d (red arrow; top) of the bead from the center of the fiber, the larger the radius ρ (white arrow; bottom) of the bright spiral-ring pattern in the proximal end. The diagonal white dashed line is the azimuthal orientation of the red vector d, which always coincides with the alignment angle of the 2 central bright points of the fingerprint patterns in the proximal end (see Fig. S10 for details). The highest LED power values measured at the distal end of the fiber (whose core is 200 µm in diameter) were around 9.5 µW, which yields an excitation intensity of 0.3 mW/mm² at the output of the fiber core, and 2.4 × 10⁻⁵mW excitation power per bead area. Exposure time: 100 ms (Miniscope FPS = 10 Hz). For more details, see Figs. S9, S10. d The concept of doing experiments with a MiniDART device, which combines a miniscope and a short implantable multimode. For future in vivo experiments, the MMF and miniscope baseplate should be glued on the mouse skull with dental cement in the same way typical miniscope experiments are performed with GRIN lenses.