Measuring the fidelity of implementation of a mental health program model

Abstract

A fidelity index of program implementation for assertive community treatment (ACT) was developed. In Study 1, 20 experts rated the importance of 73 elements proposed as critical ACT ingredients, also indicating ideal model specifications for elements. Agreement among experts on ratings of importance was high (intraclass r = .98). In Study 2, a 17-item subset of the expert-identified critical ingredients was used to construct a fidelity index with three subscales: Staffing, Organization, and Service. Internal consistencies ranged from .50 to .72, with a .81 reliability for the total scale. Fidelity was linearly related to program "generations," suggesting "program drift." In 18 ACT programs, fidelity also was associated with measures of reduction in days in psychiatric hospitals. The correlation was significant for the total scale and for the Organization and Staffing subscales but not for the Service subscale.