Distributed Temporal Coding of Visual Memory Categories in Human Hippocampal Neurons Revealed by an Interpretable Decoding Model

Abstract

The hippocampus is crucial for forming new episodic memories. While its role in encoding spatial and temporal information (where and when) is well understood, how it encodes objects (what) remains unclear due to the high dimensionality of object space. Rather than encoding each object separately, the hippocampus may encode object categories to reduce complexity. Here, an experimental-modeling approach to investigate how the hippocampus encodes visual memory categories in humans is developed. Spikes are recorded from hippocampal CA3 and CA1 neurons in 24 epilepsy patients performing a delayed match-to-sample task involving five image categories. An interpretable memory decoding model is employed to decode memory categories from hippocampal spiking activity and identify the spatio-temporal characteristics of hippocampal encoding. Using this model, the optimal temporal resolutions for decoding each visual memory category per neuron are estimated. Results indicate that visual memory categories can be decoded from hippocampal spike patterns, supporting the presence of category-specific coding. Hippocampal neuron ensembles encode memory categories in a distributed manner, akin to a population code, while individual neurons use a temporal code. Additionally, CA3 and CA1 neurons exhibit similar and redundant memory category information, likely due to strong and diffuse feedforward synaptic connections from CA3 to CA1 regions.

Keywords: human hippocampus; memory category; memory decoding model; neuronal spike; spatio‐temporal code.

Figure 1

Decoding visual memory categories from spatio‐temporal patterns of spikes recorded in the human hippocampus using an ensemble multi‐temporal‐resolution classification model. This model provides interpretable model representations of spatio‐temporal characteristics of spike patterns for encoding specific memory categories.

Figure 2

Human experimental paradigm. a) 3T MRI showing pre‐implantation hippocampal structures and post‐implantation electrode locations in one subject. The bulges visible in the probe are the location of the “macro” electrode sites. Inset: zoomed‐in view of the probe in the hippocampus. b) Layout of the micro‐macro probe containing 6 macro‐electrodes and 10 micro‐electrodes. Six and four micro‐electrodes were implanted in the CA3 and CA1 regions, respectively. c) Sample Images of the five memory categories used in the DMS task.

Figure 3

Behavioral tasks and decoding cases are designed for decoding memory category information. a) DMS task paradigm. SP: Sample Presentation; SR: Sample Response; MP: Match Presentation; MR: Match Response. b) decoding cases and control cases in the modeling.

Figure 4

Categories of sample images can be decoded from spatio‐temporal patterns of spikes recorded during Sample Response and Match Response events of a delayed DMS task in human subjects (n = 24). Pentagon plots show the decoding performance of the five memory categories in a) Sample; b) Match; and c) Time‐Shifted decoding cases. Color lines: MCCs of individual subjects; Black thick lines: average MCCs across all subjects; White shades: distributions of MCCs within categories. Note that the Label‐Shuffle control case is omitted for clarity, as it yielded zero MCC values.

Figure 5

Spatio‐temporal distributions of category information in hippocampal spike patterns are revealed by the classification model. a) peri‐event histogram of spike patterns of the five categories during the Sample Response events. b) raster plots showing a single trial of the spatio‐temporal patterns of spikes during the Sample Response event. The five categories cannot be easily distinguished in either (a) or (b). c) SCFMs of the five categories in the classification model. The red box marks the neuron shown in (e) and (f). Based on the SCFMs, this neuron contributes to encoding the five categories. d) spike counts of each category with (top panel) and without (bottom panel) using SCFMs as masks. SCFMs reveal the spatio‐temporal regions of the spike patterns that encode the category information. e) spike raster plots of trials within each category of the neuron marked in (a), (b), and (c). f) peri‐event histograms of trials of the decoded category (top panel) and other (non‐decoded) categories (bottom panel) of this neuron. Dashed lines represent the baseline firing rates. Significant differences in firing rates between decoded categories and non‐decoded categories exist in time intervals consistent with the SCFMs (bins marked with asterisks, p < 0.05).

Figure 6

Sparseness of spatio‐temporal encoding of visual memory categories during Sample Response and Match Response events of the DMS task. a) spatial sparseness of all neuron ensembles (n = 24). b) temporal sparseness of all neurons (n = 721). Bars: mean sparseness; error bars: standard deviation (STD) of sparseness. c) Contribution of temporal resolutions to the encoding of visual memory categories. Colored dots (left y‐axis): contribution of temporal resolutions to the encoding of categories in each subject. Colored lines (right y‐axis): averaged contribution of temporal resolutions across all subjects (n = 24). Black line (right y‐axis): averaged contribution of temporal resolutions across all five categories. Left: Sample Response; Right: Match Response.

Figure 7

Contribution of hippocampal CA3 and CA1 to the encoding of visual memory categories during Sample Response and Match Response events of the DMS task. a) Contributions of the CA3 and CA1 regions. Red: unique contribution of the CA3 region; Yellow: redundant contribution shared by CA3 and CA1 regions; Blue: unique contribution of the CA1 region. b) averaged contribution of hippocampal CA3 and CA1 neurons to the encoding. Each dot represents one subject (n = 24). There is no significant difference between CA3 and CA1 neurons during both Sample Response and Match Response (paired t‐test).