The paired t test and beyond: Recommendations for testing the central tendencies of two paired samples in research on speech, language and hearing pathology

Abstract

Purpose: In this tutorial we review current practice in the analysis of data obtained in designs involving two dependent samples and evaluate two conventional statistics: the t test for paired samples and its non-parametric alternative, the Wilcoxon Signed Ranks test (WSR). It is a sequel to our tutorial on the analysis of designs with two independent samples on the basis of non-count data (Rietveld & van Hout, 2015). The frequency with which these statistics are used is assessed on the basis of publications on disordered communication in Clinical Linguistics & Phonetics, Journal of Communication Disorders and Journal of Speech, Language and Hearing Research for the time interval 2006-2015. We conclude with a number of recommendations for the analysis and presentation of data.

Conclusions: Researchers should more consistently present the relevant characteristics of their data (means, medians, SD, skewness, tailedness, outliers etc.) and explicitly consider the assumptions that apply to their statistical methods, such as correlations between data obtained on two occasions, interactions between participants and treatment, and the symmetry of difference scores, many of which are hardly ever reported or even tested. Two recommendations are particularly relevant. First, the WSR is not a proper test for central tendencies as a replacement of the conventional t test for paired samples whenever assumptions about the dependent variable are in doubt. Second, researchers should choose statistical procedures on the basis of the null hypothesis (H0) to be tested and not primarily on the basis of the type of data (ordinal or interval). Two relevant H0's in the field of speech-language pathology are: (1) μ₁=μ₂ (the mean obtained in condition 1 is equal to the mean in condition 2) and (2) p=0.5, which says: the probability to obtain (for instance) higher scores in condition 2 than in condition 1 is 0.5. We recommend the permuted t test for paired samples to test the first H0 and the permuted Brunner-Munzel rank test to test the second.

Keywords: Assumptions for nonparametric statistics; Recent developments in the analysis of data obtained in paired samples designs; Statistical interaction and correlation in paired samples designs; Tests for paired samples.