Constructing quantitative implementation scales from categorical services data. Examples from a multisite evaluation

Abstract

Like measures of outcome, measures of implementation are most useful and analytically powerful when measured at client-level and are quantitative. However, high-quality, individual-level, quantitative service utilization data can be expensive, intrusive, or otherwise impractical to obtain. Cruder data--for example, presence versus absence of a given service--are often more feasible to collect, as well as more likely to be available. Consequently, evaluators can benefit by finding ways to better exploit such data at the analysis phase to compensate for shortcomings at the collection phase. This article documents one such instance in which this was done. Specifically, it describes how quantitative, client-level implementation scales were derived from qualitative (categorical) data and used to support a cross-site synthesis of implementation and outcome analyses in a multisite evaluation. It also suggests additional scenarios in which quantitative implementation scales might be derived form qualitative services data.