Meaningful change definitions: sample size planning for experimental intervention research

Abstract

Experimental tests of interventions need to have sufficient sample size to constitute a robust test of the intervention's effectiveness with reasonable precision and power. To estimate the required sample size adequately, researchers are required to specify an effect size. But what effect size should be used to plan the required sample size? Various inroads into selecting the a priori effect size have been suggested in the literature-including using conventions, prior research, and theoretical or practical importance. In this paper, we first discuss problems with some of the proposed methods of selecting the effect size for study planning. We then lay out a method for intervention researchers that provides a way out of many of these problems. The proposed method requires setting a meaningful change definition, it is specifically suited for applied researchers interested in planning tests of intervention effectiveness. We provide a hands-on walk through of the method and provide easy-to-use R functions to implement it.

Keywords: Effect size; intervention research; meaningful change definitions; practical significance; sample size planning; smallest effect size of interest.