Implemented or not implemented? Process evaluation of the school-based obesity prevention program DOiT and associations with program effectiveness

Abstract

This study investigates if and to what extent the Dutch Obesity Intervention in Teenagers (DOiT) program was implemented as intended and how this affected program effectiveness. We collected data at 20 prevocational education schools in the Netherlands. We assessed seven process indicators: recruitment, reach, dosage, fidelity, satisfaction, effectiveness and continuation. Data collection involved teacher questionnaires (n= 110), adolescent questionnaires and adiposity measures (n= 938). Using multi-level confirmatory factor analyses, we applied an innovative method to obtain explorative implementation index scores. The percentage of accomplished activities ranged from 9% (for 'closure meeting') up to 93% (for 'obtaining support within the school for adoption'). The percentage of lessons delivered decreased from 74 to 18% towards the end of the program. Fidelity to the teacher manual ranged from 85 to 26%. In general, teachers were satisfied with the DOiT lessons and teaching materials. Despite teachers' satisfaction with the DOiT lessons and teaching materials, degree of program implementation was lower than expected, especially towards the end of the program. Further, some evidence was found for an association between a higher implementation index score and program effectiveness, but more research is needed to test the validity of the implementation index.