The Neuroscience of Spatial Navigation and the Relationship to Artificial Intelligence

Abstract

Recent advances in artificial intelligence (AI) and neuroscience are impressive. In AI, this includes the development of computer programs that can beat a grandmaster at GO or outperform human radiologists at cancer detection. A great deal of these technological developments are directly related to progress in artificial neural networks-initially inspired by our knowledge about how the brain carries out computation. In parallel, neuroscience has also experienced significant advances in understanding the brain. For example, in the field of spatial navigation, knowledge about the mechanisms and brain regions involved in neural computations of cognitive maps-an internal representation of space-recently received the Nobel Prize in medicine. Much of the recent progress in neuroscience has partly been due to the development of technology used to record from very large populations of neurons in multiple regions of the brain with exquisite temporal and spatial resolution in behaving animals. With the advent of the vast quantities of data that these techniques allow us to collect there has been an increased interest in the intersection between AI and neuroscience, many of these intersections involve using AI as a novel tool to explore and analyze these large data sets. However, given the common initial motivation point-to understand the brain-these disciplines could be more strongly linked. Currently much of this potential synergy is not being realized. We propose that spatial navigation is an excellent area in which these two disciplines can converge to help advance what we know about the brain. In this review, we first summarize progress in the neuroscience of spatial navigation and reinforcement learning. We then turn our attention to discuss how spatial navigation has been modeled using descriptive, mechanistic, and normative approaches and the use of AI in such models. Next, we discuss how AI can advance neuroscience, how neuroscience can advance AI, and the limitations of these approaches. We finally conclude by highlighting promising lines of research in which spatial navigation can be the point of intersection between neuroscience and AI and how this can contribute to the advancement of the understanding of intelligent behavior.

Keywords: artificial intelligence; deep learning; learning; memory; neuroscience; reinforcement learning; spatial navigation.

Figure 1

The neuroscience of spatial navigation. (A) Key brain structures involved in rodent spatial navigation. Diagram adapted from the Allen Brain Atlas Explorer. (B) Schematic illustrating the general pattern of anatomical connectivity and the functional shift in frames of reference encoded by the brain regions that comprise the neural circuitry of spatial navigation. Hippocampus (HPC) and parahippocampal regions (entorhinal cortex, postsubiculum, and parasubiculum) encode an animal's position in space predominantly in allocentric or map-like coordinates. The blue and red boxes represent a spectrum denoting the relative density of egocentric (viewer-dependent, self-centered, or action centered frame of reference) vs. allocentric (map-like) encoding for each region. Retrosplenial cortex (RSC). Parietal cortex (PC) and anterior thalamic nucleus (ATN) are anatomically and functionally well-positioned to interface between egocentric and allocentric frames of reference within a larger navigational network. Purple boxes represent brain structures involved in value-based signals for conditional learning and spatial navigation (The basal ganglia circuit sub-diagram was inspired from Chersi and Burgess, , used with permission).

Figure 2

Neural substrates of spatial navigation. Five examples of single units that exemplify the encoding present in several regions that encompass the brain network critical for spatial navigation. (A) Example place cell recorded in hippocampus, top row is a spike/path plot, red dots represent the locations of action potentials and black lines the path of the animal. (B,C) Example grid cell and border cell recorded in parahippocampal cortex. Colormaps are standard evenly spaced colormaps and the peak firing rate is indicated. (D) Colormap for a cell in hippocampus that encodes the direction and distance of an environmental landmark. Data are from Wilber et al. (2014). (E) Polar plot showing firing rate by HD for an HD cell. Firing rate (Hz) is represented in upper right corner for each example cell. Data in (A–C,E) are from Harvey et al. (2019).

Figure 3

Models of spatial navigation and their relationship with reinforcement learning. (A) Path integration requires keeping track of the turns and distances traveled as the animal explores the environment (top). Ring attractor network model of head-direction (bottom). The idiothetic and environmental information update and rectify the spatial representations in the model. Inspired by Schultheiss and Redish (2015), used with permission. (B) Grid cells in the medial entorhinal cortex (MEC) at different scales (top) and place cells in the hippocampus (HPC) with different scales (bottom). Having access to different scales allows the system to represent space at different resolutions. Adapted from Solstad et al. (2006), used with permission. (C) Relationship between episodic and semantic memories and path integration and model-based navigation. Inspired from Buzsáki and Moser (2013). (D) Schematic representation of the deep RL approach for spatial navigation. A deep NN is used to estimate the best action to execute to maximize future rewards. (E) Example trajectories of two agents trained using place and head direction cells in Banino et al. (2018). Some of the agents developed grid-cell like representations (red) and others only place and head direction cells-like representations (blue). During learning both agents were able to reach the goal (top). During testing, when obstacles where removed, only the agents using grid-like representations used shorter routes (bottom). Diagram adapted from Banino et al. (2018), used with permission.