The psychometricians' fallacy: too clever by half?

Abstract

The psychometricians' fallacy concludes that an attribute is quantitative from the premise that it is ordinal. This fallacy occupies a central place in the paradigm of psychometrics. Most of the founders of the discipline committed it and it makes sense of otherwise anomalous developments within the discipline, such as the permissible statistics controversy and the dominant form taken by item response theories. The fallacy is displayed by showing (1) that an attribute's quantitative structure reduces to a weak order upon differences between degrees that satisfies the double cancellation, solvability, and Archimedean conditions of conjoint measurement theory and (2) the fact that any order on the degrees themselves does not entail sufficient structure on this weak order to guarantee satisfaction of these conditions. Thus, it is possible that an ordered attribute is non-quantitative. Also, each pair of differences between degrees of an ordinal attribute falls into one of two disjoint classes: (1) those where the order relation between the pair follows from an order on the attribute and (2) those where it is independent of that order and possibly diagnostic of quantitative structure and this fact means that the distinction between order and quantity is an empirical one.