Tracking sentence comprehension: Test-retest reliability in people with aphasia and unimpaired adults

Abstract

Purpose: Visual-world eyetracking is increasingly used to investigate online language processing in normal and language impaired listeners. Tracking changes in eye movements over time also may be useful for indexing language recovery in those with language impairments. Therefore, it is critical to determine the test-retest reliability of results obtained using this method.

Methods: Unimpaired young adults and people with aphasia took part in two eyetracking sessions spaced about one week apart. In each session, participants completed a sentence-picture matching task in which they listened to active and passive sentences (e.g., The [_N1+Auxwoman was] [_Vvisiting/visited] [_NP/PP2(by) the man]) and selected between two pictures with reversed thematic roles. We used intraclass correlations (ICCs) to examine the test-retest reliability of response measures (accuracy, reaction time (RT)) and online eye movements (i.e., the likelihood of fixating the target picture in each region of the sentence) in each participant group.

Results: In the unimpaired adults, accuracy was at ceiling (thus ICCs were not computed), with moderate ICCs for RT (i.e., 0.4 - 0.58) for passive sentences and low (<0.4) for actives. In individuals with aphasia, test-retest reliability was strong (0.59<ICC<0.75) for accuracy and excellent (>0.75) for RT for both sentence types. Similarly, for the unimpaired listeners, reliability of eye movements was moderate for passive sentences (NP/PP2 region) and low in all regions for active sentences. But, for the aphasic participant group, eye movement reliability was excellent for passive sentences (in the first second after sentence end) and strong for active sentences (V and NP/PP2 regions).

Conclusion: Results indicated moderate-to-low reliability for unimpaired listeners; however, reliable eye movement patterns were detected for processes specific to passive sentences (e.g., thematic reanalysis). In contrast, individuals with aphasia exhibited strong and stable performance across sentence types in response measures and online eye movements. These findings indicate that visual-world eyetracking provides a reliable measure of online sentence comprehension, and thus may be useful for investigating sentence processing changes over time.

Keywords: aphasia; eyetracking; sentence comprehension; test-retest reliability.

Figure 1

Example visual stimulus.

Figure 2

Eye movement data from one representative unimpaired participant and one representative person with aphasia. The dots indicate raw eye movement data points, i.e., the proportion of trials in which the participant was fixating the target picture, aggregated into 50 ms bins. The lines indicate the eye movement curves as modeled through local polynomial regression. TA = target advantage (i.e., the area under the eye movement curve in each region, divided by the length of the region).

Figure 3

Eye movements from each participant group, by test session and sentence type. Act = Active; Psv = Passive; S1 = Test Session 1; S2 = Test Session 2