The role of allograph representations in font-invariant letter identification

Abstract

The literate brain must contend with countless font variants for any given letter. How does the visual system handle such variability? One proposed solution posits stored structural descriptions of basic letter shapes that are abstract enough to deal with the many possible font variations of each letter. These font-invariant representations, referred to as allographs in this paper, while frequently posited, have seldom been empirically evaluated. The research reported here helps to address this gap with 2 experiments that examine the possible influence of allograph representations on visual letter processing. In these experiments, participants respond to pairs of letters presented in an atypical font in 2 tasks-visual similarity judgments (Experiment 1) and same/different decisions (Experiment 2). By using representational similarity analysis (RSA) in conjunction with linear mixed effect models (LMEM; RSA-LMEM) we show that the similarity structure of the responses to the atypical font is influenced by the predicted similarity structure of allograph representations even after accounting for font-specific visual shape similarity. Similarity due to symbolic (abstract) identity, name, and motor representations of letters are also taken into account providing compelling evidence for the unique influence of allograph representations in these tasks. These results provide support for the role of allograph representations in achieving font-invariant letter identification. (PsycINFO Database Record

Figure 1

Proposed representational levels in the processing of different fonts of upper and lower-case E and one pseudoletter. In the initial processing stages, stimuli are represented in terms of domain-general visual features (e.g. simple and complex cells representing oriented bars). Many models also posit a font-specific computed stimulus shape representations of the shape of the stimulus independent of its identity. This level of representation would allow us to describe the shape of the letter E as well as the shape of any given pseudoletter. Computed stimulus shape representations do not encode letter identity information or even whether the shape is a letter or not. These computed stimulus shape representations go on to access stored font-invariant allograph representations that presumably encode letter shapes in a manner that abstracts away from certain differences in stimulus font. The allographs in turn activate abstract or symbolic letter identity (SLI) representations that are both font and case independent—abstracting away from visual information altogether. SLI representations serve as input to the lexical and sublexical reading processes that mediate orthographic word recognition. They also serve as a conduit to cross-modal letter representations like phonological letter names and motoric production codes for written letter production.

Figure 2

Using Representational Similarity Analysis to identify the influence of specific representational types on the recognition of letters viewed in the Upright Gridfont. (A) The first column depicts the types of letter representations potentially active in response to viewing a and G in the Upright Gridfont. (B) The second column depicts an approximation of the representational content of Upright Gridfont a and G for each representational type. Computed Stimulus-Shape similarity is estimated with rotated Gridfont letters. Rotation maintains pairwise visual-spatial similarity while minimizing the identifiability of the rotated letters, thereby, limiting the influence of the other representational types on letter recognition. Allograph similarity is estimated from the similarity structure of a more typical font. Symbolic Letter identities encode font and case-invariant letter identity. Letter-name similarity is estimated from the phonetic features that compose the phonemes of the letter-names. Motoric similarity is estimated from hypothesized motor plans. (C) The third column depicts matrices of pairwise similarity estimates for each type of representation (pRSMs), characterizing the predicted similarity structures at each level of representation in response to pairs of the Upright Gridfont letters. (D) Column 4 depicts a set of similarity responses (e.g., derived from judgments or same/different RTs or accuracy) elicited in response to letter pairs (the oRSM). These similarity responses are tested for the unique influence of each of the 5 types of letter representations by running a linear mixed-effects model (LMEM) in which the 5 pRSMs simultaneously predict the responses to the Upright Gridfont letter-pairs.

Figure 3

Dissociating the influence of Computed Stimulus-Shape Representations from Allograph Representations. Participants made visual similarity judgments to letters presented in an atypical Upright Gridfont (left column), a Rotated Gridfont (middle column), and a Typical Font (right column). The average similarity judgment (computed from Experiment 1) is shown below each example letter-pair. A response of 1 indicates little or no visual similarity for the pair and 5 indicates the highest level of visual similarity. If computed stimulus-shape representations are the only type of letter representation that contributed to visual similarity judgments, we would predict similarity judgments for the Upright Gridfont (left column) and the Rotated Gridfont (middle column) to be identical. Instead, we see that the similarity judgments to the Upright Gridfont are different from similarity judgments to the Rotated Gridfont. The top row depicts an example where there is a bias to judge the Upright Gridfont letter-pair as being more similar than would be predicted by the computed stimulus-shape similarity alone while the bottom row shows the opposite effect. The fact that the Upright Gridfont similarity judgments are “pulled” in the direction of the Typical Font similarity judgments suggests an influence of allograph representations on these similarity judgments

Figure 4

Experimental stimuli. a) The novel, atypical and Upright Gridfont alphanumeric characters used as stimuli in Experiments 1. Experiment 2 used the same stimuli except the 6 stimuli corresponding to the identities [D] or [H]. b) The Rotated Gridfont stimuli used in Experiments 1 and 2. These stimuli were created by flipping the upright characters about their vertical axis and then rotating them 90°ccw. The rotation intended to render them difficult to identify. Experiment 2 did not include the 6 stimuli corresponding to rotated versions of [D] or [H]. c) Font used in the Typical Font Group. The font is Consalas which is a variant of Calibri where the width of each character is matched. d) A depiction of the square, 100 X 100 pixel grid with all of the possible features in black. Each stimulus was required to touch all 4 sides, ensuring the height and width of each stimulus were matched.

Figure 5

The group representational similarity matrices (RSMs) obtained from Experiment 1. Red cells indicate larger average similarity judgments and blue cells indicate smaller average similarity judgments. a) The group RSM for the similarity judgments to letter-pairs presented in the atypical Upright Gridfont. The Gridfont letters are depicted on the margin. The group RSM was formed by normalizing each participant’s responses to have a mean of 0 and a standard deviation of 1 and then averaging the normalized responses to each letter-pair across participants. b) The group RSM for the similarity judgments to letter-pairs presented in the atypical Rotated Gridfont. c) The group RSM for the similarity judgments to letter-pairs presented in the Typical font.

Figure 6

Example trial in the same-different decision paradigm.

Figure 7

The group representational similarity matrices (RSMs) computed from the RTs to different trials in Experiment 2. Red cells indicate slower average RTs (more similarity) and blue cells indicate faster RTs (less similarity). a) The group RSM for the RTs to letter-pairs presented in the Upright Gridfont. The Gridfont letters are depicted on the margin. The group RSM was formed by normalizing each participant’s responses to have a mean of 0 and a standard deviation of 1 and then averaging the normalized responses to each letter-pair across participants. b) The group RSM for the average normalized RTs to letter-pairs presented in the Rotated Gridfont.

Figure 8

The group representational similarity matrices (RSMs) depicting the error counts to different trials across participants in Experiment 2. Errors were defined as “same” responses to “different” trials. Darker cells indicate a greater error count. a) The group RSM for the RTs to letter-pairs presented in the Upright Gridfont. The Gridfont letters are depicted on the margin. The group RSM was formed by normalizing each participant’s responses to have a mean of 0 and a standard deviation of 1 and then averaging the normalized responses to each letter-pair across participants. b) The group RSM for the average normalized RTs to letter-pairs presented in the Rotated Gridfont.