Quasi-experimental study designs series-paper 9: collecting data from quasi-experimental studies

Abstract

Objective: To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs.

Study design and setting: All quasi-experimental (QE) designs.

Results: When designing a systematic review of QE studies, potential sources of heterogeneity-both theory-based and methodological-must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls used are viewed as of greatest importance. Potential sources of bias and confounding are also addressed.

Conclusion: Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis.

Keywords: Bivariate effect size; Effect modifiers; Meta-analysis; Moderator variables; Partial effect size; Quasi-experiment.