Analogy-Related Information Can Be Accessed by Simple Addition and Subtraction of fMRI Activation Patterns, Without Participants Performing any Analogy Task

Abstract

Analogical reasoning, for example, inferring that teacher is to chalk as mechanic is to wrench, plays a fundamental role in human cognition. However, whether brain activity patterns of individual words are encoded in a way that could facilitate analogical reasoning is unclear. Recent advances in computational linguistics have shown that information about analogical problems can be accessed by simple addition and subtraction of word embeddings (e.g., wrench = mechanic + chalk - teacher). Critically, this property emerges in artificial neural networks that were not trained to produce analogies but instead were trained to produce general-purpose semantic representations. Here, we test whether such emergent property can be observed in representations in human brains, as well as in artificial neural networks. fMRI activation patterns were recorded while participants viewed isolated words but did not perform analogical reasoning tasks. Analogy relations were constructed from word pairs that were categorically or thematically related, and we tested whether the predicted fMRI pattern calculated with simple arithmetic was more correlated with the pattern of the target word than other words. We observed that the predicted fMRI patterns contain information about not only the identity of the target word but also its category and theme (e.g., teaching-related). In summary, this study demonstrated that information about analogy questions can be reliably accessed with the addition and subtraction of fMRI patterns, and that, similar to word embeddings, this property holds for task-general patterns elicited when participants were not explicitly told to perform analogical reasoning.

Keywords: fMRI; language; word analogy; word2vec.

Figure 1.

Schematics of the current study. (A) Similar to solving analogy questions, information about word analogies was accessed by translating word relations from one context to another. (B) The categorical and thematic membership of each word can be determined by applying simple word arithmetic operations to such features. (C) We attempted to study the simple yet unanswered question: Can such analogy-related information be accessed from applying addition and subtraction to the fMRI patterns of individual words?

Figure 2.

Description of the ranking metrics. Top: The example analogy question and the arithmetic operations we applied to create the predicted fMRI pattern (i.e., wrench ^pred ). The target word (i.e., wrench) is underlined. Bottom: The three main ranking metrics. For each metric, the Pearson correlation coefficient between wrench ^pred and the target pattern (in bold font) was ranked against the correlation between wrench ^pred and all other candidate words. The rank was scaled to [0, 1] and averaged across all four target words in all possible analogy questions. In the identity metric the target pattern was the fMRI pattern of the target word (i.e., wrench), and all other words in the stimuli list, except the four words in the analogy question, were candidate words. In the category metric, category templates, which were the averaged fMRI patterns across all remaining words in each category, were used instead of the fMRI patterns of individual words. The target category template was the category that the word actually belonged to, and the two other templates served as candidates. In the theme metric, the target pattern was the word in the same theme and the final unused category (i.e., garage), and all other words in the unused category (i.e., building words) were candidate words.

Figure 3.

Nine prespecified semantic related ROIs. Colors denote different ROIs and not the significance of results.

Figure 4.

Information about the identity (e.g., wrench vs. 41 other words), category (e.g., tool vs. two other categories), theme (e.g., repairing vs. all 13 other themes) of a word can be accessed from the addition and subtraction of fMRI patterns of the three other words in an analogy question. The identity of the target word can also be distinguished from words in its category (4th column) and a foil word from its theme (5th column). Error bars represent the standard error of the mean across participants (* p < 0.05; ** p < 0.01; *** p < 0.001). The dotted line represented chance-level performance (mean rank = 0.5) derived from the participant-level permutation process. Black dots represent the mean rank of each participant. This figure only displays results from fMRI patterns where 100 voxels were selected according to the voxel selection procedure (see Materials and Methods), and they do not need to be spatially connected; see Supp. Table 1 for the full results.

Figure 5.

Information about the identity (e.g., wrench vs. 41 other words), category (e.g., tool vs. two other categories), and theme (e.g., repairing vs. all 13 other themes) of a word can be accessed from the addition and subtraction of fMRI patterns in the left supramarginal gyrus. The identity of the target word can also be distinguished within its category (4th column) and from a close foil word in its theme (5th column). Error bars represented the standard error of the mean across participants (* p < 0.05). The dotted line represented chance-level performance (mean rank = 0.5). Black dots represent the mean rank of each participant.