Analysis of pilot and early phase studies with small sample sizes

Abstract

The need for statistical methodologies for analysing a small size study, such as a pilot or so-called 'proof of concept' study, has not been paid much attention in the past. Recently the Institute of Medicine (IOM) formed a committee and held a workshop to discuss methodologies for conducting clinical trials with small number participants. In this paper we argue that the hypothesis of treatment effect in a small pilot study should be set up to test whether any individual subject has an effect rather than whether the group mean or median has shifted as often done for large, confirmatory clinical trials. Based on this paradigm we propose multiple test procedures as one option when individuals have enough observations, and a mixture-distribution approach when individuals have one or more observations. The latter approach may be used in either a one- or two-group setting, and is our focus in this paper. We present the likelihood ratio tests for the mixture models. Examples are given to demonstrate the methods.