A tool for efficient and accurate segmentation of speech data: announcing POnSS

Abstract

Despite advances in automatic speech recognition (ASR), human input is still essential for producing research-grade segmentations of speech data. Conventional approaches to manual segmentation are very labor-intensive. We introduce POnSS, a browser-based system that is specialized for the task of segmenting the onsets and offsets of words, which combines aspects of ASR with limited human input. In developing POnSS, we identified several sub-tasks of segmentation, and implemented each of these as separate interfaces for the annotators to interact with to streamline their task as much as possible. We evaluated segmentations made with POnSS against a baseline of segmentations of the same data made conventionally in Praat. We observed that POnSS achieved comparable reliability to segmentation using Praat, but required 23% less annotator time investment. Because of its greater efficiency without sacrificing reliability, POnSS represents a distinct methodological advance for the segmentation of speech data.

Keywords: Segmentation; Speech data.

Fig. 1

A diagrammatic representation of the annotation process. See the text for full details

Fig. 2

Screenshots of the browser interfaces for the orthographic transcription (left), triage (middle), and retrimming tasks (right) in POnSS

Fig. 3

Panel a: the observed distributions of the difference between segmented times and the median segmentation for each word, for POnSS and manual annotation modalities (colors). Panel b: an example of the optimized mixture-model fit (orange) to the observed distribution of one of the samples (black line). Panel c: Solid violins show the posteriors of Model 1 (see text) for the effect of modality on the sigma, with median (points), 95% HDIs (highest density intervals, thin black lines) and 66% HDIs (thick black lines)

Fig. 4

Distributions of bootstrap-resampled estimates of how many annotator hours it would take to yield 5000 well-segmented words by the two modalities (translucent violins). Overlaid are solid violins showing the posteriors of Model 2 for the effect of modality, with median (points), 95% and 66% HDIs are too narrow to see in the figure