Evaluating Intervention Programs with a Pretest-Posttest Design: A Structural Equation Modeling Approach

Abstract

A common situation in the evaluation of intervention programs is the researcher's possibility to rely on two waves of data only (i.e., pretest and posttest), which profoundly impacts on his/her choice about the possible statistical analyses to be conducted. Indeed, the evaluation of intervention programs based on a pretest-posttest design has been usually carried out by using classic statistical tests, such as family-wise ANOVA analyses, which are strongly limited by exclusively analyzing the intervention effects at the group level. In this article, we showed how second order multiple group latent curve modeling (SO-MG-LCM) could represent a useful methodological tool to have a more realistic and informative assessment of intervention programs with two waves of data. We offered a practical step-by-step guide to properly implement this methodology, and we outlined the advantages of the LCM approach over classic ANOVA analyses. Furthermore, we also provided a real-data example by re-analyzing the implementation of the Young Prosocial Animation, a universal intervention program aimed at promoting prosociality among youth. In conclusion, albeit there are previous studies that pointed to the usefulness of MG-LCM to evaluate intervention programs (Muthén and Curran, 1997; Curran and Muthén, 1999), no previous study showed that it is possible to use this approach even in pretest-posttest (i.e., with only two time points) designs. Given the advantages of latent variable analyses in examining differences in interindividual and intraindividual changes (McArdle, 2009), the methodological and substantive implications of our proposed approach are discussed.

Keywords: experimental design; intervention; latent variables; multiple group latent curve model; pretest-posttest; second order latent curve model; structural equation modeling.

Figure 1

Second Order Latent Curve Models with parallel indicators (i.e., residual variances of observed indicators are equal within the same latent variable: ε₁ within η₁and ε₂ within η₂). All the intercepts of the observed indicators (Y) and endogenous latent variables (η) are fixed to 0 (not reported in figure). In model A, the residual variances of η₁ and η₂ (ζ₁ and ζ₂, respectively) are freely estimated, whereas in Model B they are fixed to 0. ξ₁, intercept; ξ₂, slope; κ₁, mean of intercept; κ₂, mean of slope; ϕ₁, variance of intercept; ϕ₂, variance of slope; ϕ₁₂, covariance between intercept and slope; η₁, latent variable at T1; η₂, latent variable at T2; Y, observed indicator of η; ε, residual variance/covariance of observed indicators.

Figure 2

Best fitting Second Order Multiple Group Latent Curve Model with parameter estimates for both groups. Parameters in bold were fixed. This model has parallel indicators (i.e., residual variances of observed indicators are equal within the same latent variable, in each group). All the intercepts of the observed indicators (Y) and endogenous latent variables (η) are fixed to 0 (not reported in figure). G1, intervention group; G2, control group; ξ₁, intercept of prosociality; ξ₂, slope of prosociality; η₁, prosociality at T1; η₂, prosociality at T2; Y, observed indicator of prosociality; ε, residual variance of observed indicator. ^n.s. p > 0.05; ^*p < 0.05; ^**p < 0.01; ^***p < 0.001.

Figure 3

Trajectories of prosocial behavior for intervention group (G1) and control group (G2) in the best fitting model (Model 2 in Table 2).