Student t-tests for potentially abnormal data

Abstract

When the one-sample or two-sample t-test is either taught in the class room or applied in practice to small samples, there is considerable divergence of opinion as to whether or not the inferences drawn are valid. Many point to the 'Robustness' of the t-test to violations of assumptions, while others use rank or other robust methods because they believe that the t-test is not robust against violations of such assumptions. It is quite likely, despite the apparent divergence of these two opinions, that both arguments have considerable merit. If we agree that this question cannot possibly be resolved in general, the issue becomes one of determining, before any actual data have been collected, whether the t-test will or will not be robust in a specific application. This paper describes statistical analysis system software, covering a large collection of potential input probability distributions, to investigate both the null and power properties of various one- and two-sample t-tests and their normal approximations, as well as the Wilcoxon two-sample and sign-rank one-sample tests, allowing potential practitioners to determine, at the study design stage, whether the t-test will be robust in their specific application. Sample size projections, based on these actual distributions, are also included. This paper is not intended as a tool to assess robustness after the data have been collected.