Measuring and modelling the quality of 40 post-disaster mental health and psychosocial support programmes

Abstract

Disasters can have an enormous impact on the health and well-being of those affected. Internationally, governments and service providers are often challenged to address complex psychosocial problems. Ideally, the potentially broad range of support activities include a coherent, high-quality mental health and psychosocial support (MHPSS) programme. We present a theory-driven quantitative analysis of the quality of 40 MHPSS programmes, mostly implemented in European disaster settings. The objective is to measure quality domains recognized as relevant in the literature and to empirically test associations. During the EU project "Operationalizing Psychosocial Support in Crisis" (OPSIC) an evaluation survey was designed and developed for this purpose and completed by 40 MHPSS programme coordinators involved in different mass emergencies and disasters. We analysed the survey data in two steps. Firstly, we used the data to operationalize quality domains of a MHPSS programme, tested constructs and assessed their internal consistency reliability. A total of 26 out of 44 survey items clustered into three of the four domains identified within the theoretical framework: "planning and delivery system" (Cronbach's alpha 0.82); "general evaluation criteria" (Cronbach's alpha 0.82); and "essential psychosocial principles" (Cronbach's alpha 0.75). "Measures and interventions applied", theoretically a potential fourth domain, could not be confirmed to empirically cluster together. Secondly, several models with associations between domains and measures and interventions were tested and compared. The model with the best fit suggests that in MHPSS programmes with a higher planning and delivery systems score, a larger number of measures and interventions from evidence-informed guidelines are applied. In such programmes, coordinators are more positive about general evaluation criteria and the realization of essential psychosocial principles. Moreover, the analyses showed that some measures and interventions are more likely to be applied in programmes with more evolved planning and delivery systems, yet for most measures and interventions the likelihood of being applied is not linked to planning and delivery system status, nor to coordinator perceptions concerning psychosocial principles and evaluation criteria. Further research is necessary to validate and expand the findings and to learn more about success factors and obstacles for MHPSS programme implementation.

Fig 1. Three test models.

The three test models display associations between the quality domains of a programme. In model A a more developed planning and delivery system is accompanied by a higher score on measures and interventions applied (relation a), resulting in higher perceived scores assigned to essential psychosocial principles (relation b) and general evaluation criteria (relation c). Model B follows the same line of reasoning but now the planning and delivery system status directly influences perceived essential psychosocial principles (relation d) and general evaluation criteria scores as well (relation e). Model C is restricted to the influence of a change in planning and delivery score on the other domains.