Do evidence-based interventions work when tested in the "real world?" A systematic review and meta-analysis of parent management training for the treatment of child disruptive behavior

Abstract

Evidence-based interventions are often unavailable in everyday clinical settings. This may partly reflect practitioners' assumptions that research evidence does not reflect "real-world" conditions. To examine this further, we systematically assessed the clinical effectiveness of parent management training (PMT) for the treatment of child disruptive behavior across different real-world practice contexts. We identified 28 relevant randomized controlled trials from a systematic search of electronic bibliographic databases and conducted a meta-analysis of child outcomes across trials. Planned subgroup analyses involved comparisons between studies grouped according to individual real-world practice criteria and total real-world practice criteria scores, reflecting the extent to which PMT was delivered by non-specialist therapists, to a clinic-referred population, in a routine setting, and as part of a routine service. Meta-analysis revealed a significant overall advantage for PMT compared with waitlist control conditions. Subgroup analyses did not demonstrate significant differences in effect size estimates according to the total number of real-world practice criteria met by studies. Moreover, no consistent relationships were found between specific practice criteria and effect size estimates. In conclusion, PMT appears to be an effective treatment for children with disruptive behavior problems. There was no clear evidence that conducting PMT in real-world practice contexts is a deterrent to achieving effective child behavior outcomes, although relative advantage to "usual care" was not directly examined and the power of the analysis was limited as a result of significant heterogeneity. More research is needed to investigate whether this finding is generalizable to other psychological interventions. Suggestions are also made for developing more differentiated criteria to assist with evaluating the specific applicability of research evidence to different care providers.