Evaluation of the Hierarchical Correspondence between the Human Brain and Artificial Neural Networks: A Review

Abstract

Artificial neural networks (ANNs) that are heavily inspired by the human brain now achieve human-level performance across multiple task domains. ANNs have thus drawn attention in neuroscience, raising the possibility of providing a framework for understanding the information encoded in the human brain. However, the correspondence between ANNs and the brain cannot be measured directly. They differ in outputs and substrates, neurons vastly outnumber their ANN analogs (i.e., nodes), and the key algorithm responsible for most of modern ANN training (i.e., backpropagation) is likely absent from the brain. Neuroscientists have thus taken a variety of approaches to examine the similarity between the brain and ANNs at multiple levels of their information hierarchy. This review provides an overview of the currently available approaches and their limitations for evaluating brain-ANN correspondence.

Keywords: artificial neural networks; hierarchical correspondence; neuroscience.

Figure 1

Four levels of correspondence evaluation. (a) The node level evaluates the correspondence between one node of the ANN and the smallest measured unit of the brain (e.g., a single unit recording, an electrode, a voxel). When computing correspondence via encoding, brain units are modeled using ANN nodes, and vice versa in decoding. (b) The layer level evaluates the correspondence between an ANN hidden layer and a brain region. Illustrated approaches show encoding via many nodes of a layer to a single voxel activity in a known brain region (top), decoding via many voxels to one node (middle), and correspondence between many nodes and many voxels via RSA (representational similarity analysis) (bottom). (c) The network level evaluates the correspondence between the overall information flow inside an ANN and across multiple regions, cortices, or the whole brain. Illustrated approaches sum the number of voxels in each region most associated with a given ANN layer (top), and examine the relationship between voxels’ scores (e.g., principal gradient score [26]) and their associated layers (bottom). (d) Behavioral level correspondence compares the ANN output, which includes qualitative assessments such as the Turing test in conversation, or compares the performance of behavioral metrics (e.g., classification accuracy, response time, error pattern, playing games) against their human counterparts. Brain illustrations were generated using the Connectome Workbench visualization software v1.4.2 [41] with Gordon’s parcellation [42].

Figure 1

Four levels of correspondence evaluation. (a) The node level evaluates the correspondence between one node of the ANN and the smallest measured unit of the brain (e.g., a single unit recording, an electrode, a voxel). When computing correspondence via encoding, brain units are modeled using ANN nodes, and vice versa in decoding. (b) The layer level evaluates the correspondence between an ANN hidden layer and a brain region. Illustrated approaches show encoding via many nodes of a layer to a single voxel activity in a known brain region (top), decoding via many voxels to one node (middle), and correspondence between many nodes and many voxels via RSA (representational similarity analysis) (bottom). (c) The network level evaluates the correspondence between the overall information flow inside an ANN and across multiple regions, cortices, or the whole brain. Illustrated approaches sum the number of voxels in each region most associated with a given ANN layer (top), and examine the relationship between voxels’ scores (e.g., principal gradient score [26]) and their associated layers (bottom). (d) Behavioral level correspondence compares the ANN output, which includes qualitative assessments such as the Turing test in conversation, or compares the performance of behavioral metrics (e.g., classification accuracy, response time, error pattern, playing games) against their human counterparts. Brain illustrations were generated using the Connectome Workbench visualization software v1.4.2 [41] with Gordon’s parcellation [42].

Figure 2

(a) Principal gradient (PG) of connectivity spans the brain, with lower scores at the start of the PG occupying sensorimotor regions and higher scores occupying higher cognitive and associative regions in a generally posterior-to-anterior gradient. (b) The distribution of brain functions along the PG begins at sensorimotor functions that converge transmodally toward higher cognition and affective processing. Figures adapted from [26].