Inter-rater reliability in classification of canonical babbling status based on canonical babbling ratio in infants with isolated cleft palate randomised to Timing of Primary Surgery for Cleft Palate (TOPS)

Abstract

Canonical babbling (CB) is commonly defined as present when at least 15% of all syllables produced are canonical, in other words a canonical babbling ratio (CBR) ≥0.15. However, there is limited knowledge about inter-rater reliability in classification of CB status based on CBR and inter-rater differences in assessment of CBR. We investigated inter-rater reliability of experienced Speech Language Therapists (SLTs) on: classification of CB status based on CBR ≥ 0.15, CBRs and the total number of syllables per infant used to calculate CBR.Each infant (n = 484) was video-recorded at a clinical site in play interaction with their parent as part of the randomised controlled trial Timing of Primary Surgery for Cleft Palate. Each recording was subsequently assessed by three independent SLTs, from a pool of 29 SLTs. They assessed the recordings in real time.The three assessing SLTs agreed in classification of CB status in 423 (87.4%) infants, with higher complete agreement for canonical (91%; 326/358) than non-canonical (77%; 97/126). The average difference in CBR and total number of syllables identified between the SLT assessments of each infant was 0.12 and 95, respectively.This study provided new evidence that one trained SLT can reliably classify CB status (CBR ≥ 0.15) in real time when there is clear distinction between the observed CBR and the boundary (0.15); however, when the observed CBR approaches the boundary multiple SLT assessments are beneficial. Thus, we recommend to include assessment of inter-rater reliability, if the purpose is to compare CBR and total syllable count across infants or studies.Trial registration number here: www.clinicaltrials.gov, identifier NCT00993551.

Keywords: Canonical babbling status; agreement; assessment variation; canonical babbling ratio; isolated cleft palate.