Developing and psychometric testing of the evaluation tool for disaster exercises design stage: A mixed method study

Abstract

Background: Exercise in different health sectors is an important step in preparing programs for disaster risk management. The present study aimed to develop and validate a tool for evaluating disaster exercises during the design stage in the health sector.

Methods: This methodological study was conducted in two phases using an explanatory sequential mixed method approach. Semi-structured interviews with 25 disaster-related health professionals were conducted during the qualitative phase (item generation), and a systematic review was done to evaluate items pool of disaster exercises design stage tool. The quantitative phase (item reduction) assessed both face and content validity, as well as reliability using Cronbach's alpha and intra-class correlation coefficient.

Results: At the first stage four main categories and eleven subcategories were extracted from the data, the main categories including "coordination, command and guidance of exercise", "hardware and software requirements of exercise ", "organizational exercise resources", and "communication and exercise public information". The initial items pool included 164 items that were reduced to 50 after the assessment of validity (face and content). Cronbach's alpha (0.89) and intra-class correlation coefficient (ICC = 0.72) tests indicated that the tool is also reliable.

Conclusion: The research findings provide a new categorization perspective to understand the disaster exercises evaluation in the health sector. The existing 50-item tool can evaluate disaster exercises design stage in the health sector through collecting data with appropriate validity and reliability.