Identification of children's stuttered and nonstuttered speech by highly experienced judges: binary judgments and comparisons with disfluency-types definitions

Abstract

Purpose: The purposes of this study were (a) to determine whether highly experienced clinicians and researchers agreed with each other in judging the presence or absence of stuttering in the speech of children who stutter and (b) to determine how those binary stuttered/nonstuttered judgments related to categorizations of the same speech based on disfluency-types descriptions of stuttering.

Method: Eleven highly experienced judges made binary judgments of the presence or absence of stuttering for 600 audiovisually recorded 5-s speech samples from twenty 2- to 8-year-old children who stuttered. These judgments were compared with each other and with disfluency-types judgments in multiple interval-by-interval assessments and by using multiple definitions of agreement.

Results: Interjudge agreement for the highly experienced judges in the binary stuttered/nonstuttered task varied from 39.0% to 89.1%, depending on methods and definitions used. Congruence between binary judgments and categorizations based on disfluency types also varied depending on methods and definitions, from 21.6% to 100%.

Conclusions: Agreement among highly experienced judges, and congruence between their binary judgments of stuttering and categorizations based on disfluency types, were relatively high using some definitions and very low using others. These results suggest the use of measurement methods other than those based on disfluency types for quantifying or describing children's stuttering. They also suggest both the need for, and potential methods for, training to increase judges' accuracy and agreement in identifying children's stuttering.