Comparing perceptual category learning across modalities in the same individuals

Abstract

Category learning is a fundamental process in human cognition that spans the senses. However, much still remains unknown about the mechanisms supporting learning in different modalities. In the current study, we directly compared auditory and visual category learning in the same individuals. Thirty participants (22 F; 18-32 years old) completed two unidimensional rule-based category learning tasks in a single day - one with auditory stimuli and another with visual stimuli. We replicated the results in a second experiment with a larger online sample (N = 99, 45 F, 18-35 years old). The categories were identically structured in the two modalities to facilitate comparison. We compared categorization accuracy, decision processes as assessed through drift-diffusion models, and the generalizability of resulting category representation through a generalization test. We found that individuals learned auditory and visual categories to similar extents and that accuracies were highly correlated across the two tasks. Participants had similar evidence accumulation rates in later learning, but early on had slower rates for visual than auditory learning. Participants also demonstrated differences in the decision thresholds across modalities. Participants had more categorical generalizable representations for visual than auditory categories. These results suggest that some modality-general cognitive processes support category learning but also suggest that the modality of the stimuli may also affect category learning behavior and outcomes.

Keywords: Audition; Category learning; Modality effects; Vision.

Figure 1.

Task procedures for category training and generalization test, across the two modalities. Stimulus distributions for auditory (left) and visual (right) for the category training (circle) and generalization test (x), with optimal distinction between the categories shown as a dashed line.

Figure 2.

Drift-diffusion model for perceptual decision making. After an initial amount of time δ_s required to encode an input signal s, the evidence in favor of a response category d accumulates according to a Wiener diffusion process with drift μ_d,s. The decision d is eventually taken if the underlying process is the first to reach its decision threshold b_d,s. The curves represent the probability distributions of the response times corresponding to the two possible decisions d ∈ {1,2}.

Figure 3.

A. Accuracy during category training for the auditory and visual tasks relative to chance (dashed line at 50% accuracy) in Experiment 1 (top) and Experiment 2 (bottom). Lighter lines show individual performance and darker lines show the mean. Error bars reflect standard error of the mean. B-C. Correlations between accuracy in the auditory task and visual task, with a logit transform (B) in the first block and (C) in the final block. D-E. Population estimates of (D) evidence accumulation rates and (E) decision thresholds from the drift-diffusion models in the auditory and visual tasks. Error bars reflect 95% credible intervals around population estimates.

Figure 4.

A. Accuracy in the generalization test for the auditory and visual tasks relative to chance (dashed line at 50% accuracy), with performance for individuals connected with gray lines with Experiment 1 on top and Experiment 2 on bottom. B. Categorization curve as a function of the placement of the stimulus along the category-relevant dimension for the auditory and visual tasks, where the dashed line at 50% reflects an equal response of Category A and Category B for a given category-relevant dimension value. C. Correlation between generalization accuracy in the auditory task and visual task, with a logit transform.