Statistical difficulties of detecting interactions and moderator effects

Abstract

Although interaction effects are frequently found in experimental studies, field researchers report considerable difficulty in finding theorized moderator effects. Previous discussions of this discrepancy have considered responsible factors including differences in measurement error and use of nonlinear scales. In this article we demonstrate that the differential efficiency of experimental and field tests of interactions is also attributable to the differential residual variances of such interactions once the component main effects have been partialed out. We derive an expression for this residual variance in terms of the joint distribution of the component variables and explore how properties of the distribution affect the efficiency of tests of moderator effects. We show that tests of interactions in field studies will often have less than 20% of the efficiency of optimal experimental tests, and we discuss implications for the design of field studies.