Community-Supported Shared Infrastructure in Support of Speech Accessibility

Abstract

Purpose: The Speech Accessibility Project (SAP) intends to facilitate research and development in automatic speech recognition (ASR) and other machine learning tasks for people with speech disabilities. The purpose of this article is to introduce this project as a resource for researchers, including baseline analysis of the first released data package.

Method: The project aims to facilitate ASR research by collecting, curating, and distributing transcribed U.S. English speech from people with speech and/or language disabilities. Participants record speech from their place of residence by connecting their personal computer, cell phone, and assistive devices, if needed, to the SAP web portal. All samples are manually transcribed, and 30 per participant are annotated using differential diagnostic pattern dimensions. For purposes of ASR experiments, the participants have been randomly assigned to a training set, a development set for controlled testing of a trained ASR, and a test set to evaluate ASR error rate.

Results: The SAP 2023-10-05 Data Package contains the speech of 211 people with dysarthria as a correlate of Parkinson's disease, and the associated test set contains 42 additional speakers. A baseline ASR, with a word error rate of 3.4% for typical speakers, transcribes test speech with a word error rate of 36.3%. Fine-tuning reduces the word error rate to 23.7%.

Conclusions: Preliminary findings suggest that a large corpus of dysarthric and dysphonic speech has the potential to significantly improve speech technology for people with disabilities. By providing these data to researchers, the SAP intends to significantly accelerate research into accessible speech technology.

Supplemental material: https://doi.org/10.23641/asha.27078079.

Figure 1.

Best published word error rates (%), up to each specified year, of automatic speech recognizers tested on standard, widely available corpora of dysarthric speech (orange, top curve) and nondysarthric speech (blue, bottom curve). ASR = automatic speech recognition.

Figure 2.

The differential diagnostic pattern dimensions in the study of Darley et al. (1969) are a set of 41 Likert scales, each describing a dimension of disability audible in an utterance. A total of 5,342 utterances in the 2023-10-05 SAP data package (an average of 25.3 per participant) were rated using differential diagnostic pattern dimensions. This figure shows histograms of those 5,342 utterances, as rated using 10 of the differential diagnostic pattern dimensions.

Figure 3.

Examples of utterances with low, moderate, and high intelligibility loss. (a) Utterance with low intelligibility loss, but highly breathy voice: “My favorite book … ” (b) Utterance with moderate intelligibility loss, glottalized on the first word: “Set an alarm … ” (c) Utterance with high intelligibility loss: “How's the traffic … ” Note the drop in loudness by 20 dB following the first syllable, the sonorance of the two fricatives, and the reduction of the /r/ in “traffic” to a /w/. (d) Utterance with high intelligibility loss and high prevalence of repeated phonemes: “Create a (g-) grocery sh-.” Note approximately 10 attempts to produce the /g/ in grocery, from 1.56 s to 2.50 s.

Figure 4.

Examples of unusually fast and unusually slow speech. (a) Unusually fast, hypoarticulated speech: “That is the way that they keep elephants at the circus, you know” (16 canonical syllables, 14 produced syllables, 2 s). (b) Unusually slow, creaky speech: “Set a reminder … ” (four syllables in 2 s).

Figure 5.

Average word error rate on the unshared prompts spoken by each person in the test corpus, plotted as a function of the number of words in the corresponding reference transcript. WER = word error rate.