The heterogeneity of mental representation: Ending the imagery debate

Abstract

The possible ways that information can be represented mentally have been discussed often over the past thousand years. However, this issue could not be addressed rigorously until late in the 20th century. Initial empirical findings spurred a debate about the heterogeneity of mental representation: Is all information stored in propositional, language-like, symbolic internal representations, or can humans use at least two different types of representations (and possibly many more)? Here, in historical context, we describe recent evidence that humans do not always rely on propositional internal representations but, instead, can also rely on at least one other format: depictive representation. We propose that the debate should now move on to characterizing all of the different forms of human mental representation.

Keywords: artificial intelligence; imagery debate; mental codes; mental imagery; working memory.

Fig. 1.

Depictive visual features in the brain. (A) Position in visual space in the world is processed depictively in the brain. (B) Depictive visual features used by a voxel-wise model to decode mental images in a study by Naselaris et al. (13). (C) Spatial orientation is processed in a depictive manner in primate visual cortex, modified from a study by Blasdel and Salama (15). (D) Behavioral data from a study by Pearson et al. (16) showing how the content of a mental image biases later perception in an orientation (depictive)-specific manner. deg, degree; LE, left eye; RE, right eye.