The relationships among design experiments, invariant measurement scales, and domain theories

Abstract

In this paper we discuss principled design experiments, a rigorous, experimentally-oriented form of design-based research. We show the dependence of design experiments on invariant measurement scales. We discuss four kinds of invariance culminating in interpretive invariance, and how this in turn depends on increasingly adequate theories of a domain. These theories give an account of the dimensions and ordered attainments on a set of dimensions that span a domain appropriately. This account may be called a domain theory or learning theory of progressive attainments (in a local domain). We show the direct, and the broader benefits of developing and using these descriptive theories of a domain to guide prescriptive design approaches to research. In process of giving an account of this set of interdependencies, we will discuss aspects of the design method we are using, called Validity-Centered Design. This design framework guides the development of instruments based on domain theories, the development of learning opportunities; also based on domain theories, and the construction of a sound validity argument for systems that integrate learning with assessment.