Development of a formative assessment tool for measurement of performance in multi-professional resuscitation teams

Abstract

Background: Treating cardiac arrest is linked to the mutual performance of several health-care individuals' task coordination. Non-technical skills, including communication, leadership and team interaction, could improve sequencing the tasks in the cardiac arrest algorithm. Non-technical skills have been a part of crew resource management training, created to improve safety in aviation. This study aimed, first, to establish crew resource management and non-technical skill-based learning objectives and behavioural markers for the performance of multi-professional resuscitation teams; second, to develop a checklist and to evaluate the validity and reliability of the checklist; and, finally, to develop a simulation-based course including the checklist on behavioural markers, as a tool for learning and assessment.

Method: A seven-step procedure was used. Findings from interviews with Advanced Life Support instructors and analysis of critical incidents were used to create learning objectives, assessment tools and course curriculum. Reliability and validity were tested by assessing digital versatile disc (DVD)-recorded simulated cardiac arrests.

Results: A checklist with 22 behavioural markers based on nine learning objectives was developed and embedded in an 8-h full-scale simulation course. Inter-rater reliability of the checklist (intra-class correlation) was 0.9. Concurrent validity (intra-class correlation) was 0.93. Rate of agreement (0.58-0.91) and kappa values (0.03-0.82) on single items varied.

Conclusion: A full-scale simulation course and a checklist with 22 behavioural markers were developed. Good inter-rater reliability and concurrent validity of the checklist were demonstrated. Single items on the checklist need refinement to improve accuracy.