Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2021 Mar 25;21(1):184.
doi: 10.1186/s12909-021-02580-4.

Development of a novel global rating scale for objective structured assessment of technical skills in an emergency medical simulation training

Andreas Zoller  1 , Tobias Hölle  2 , Martin Wepler  3   4 , Peter Radermacher  3   4 , Benedikt L Nussbaum  4
Affiliations

Development of a novel global rating scale for objective structured assessment of technical skills in an emergency medical simulation training

Andreas Zoller et al. BMC Med Educ. .

Abstract

Background: Medical simulation trainings lead to an improvement in patient care by increasing technical and non-technical skills, procedural confidence and medical knowledge. For structured simulation-based trainings, objective assessment tools are needed to evaluate the performance during simulation and the learning progress. In surgical education, objective structured assessment of technical skills (OSATS) are widely used and validated. However, in emergency medicine and anesthesia there is a lack of validated assessment tools for technical skills. Thus, the aim of the present study was to develop and validate a novel Global Rating Scale (GRS) for emergency medical simulation trainings.

Methods: Following the development of the GRS, 12 teams of different experience in emergency medicine (4th year medical students, paramedics, emergency physicians) were involved in a pre-hospital emergency medicine simulation scenario and assessed by four independent raters. Subsequently, interrater reliability and construct validity of the GRS were analyzed. Moreover, the results of the GRS were cross-checked with a task specific check list. Data are presented as median (minimum; maximum).

Results: The GRS consists of ten items each scored on a 5-point Likert scale yielding a maximum of 50 points. The median score achieved by novice teams was 22.75 points (17;30), while experts scored 39.00 points (32;47). The GRS overall scores significantly discriminated between student-guided teams and expert teams of emergency physicians (p = 0.005). Interrater reliability for the GRS was high with a Kendall's coefficient of concordance W ranging from 0.64 to 0.90 in 9 of 10 items and 0.88 in the overall score.

Conclusion: The GRS represents a promising novel tool to objectively assess technical skills in simulation training with high construct validity and interrater reliability in this pilot study.

Keywords: Emergency medicine; Global rating scale; Objective structured assessment; Simulation; Technical skills.

PubMed Disclaimer

Conflict of interest statement

The authors declare no conflicts of interest.

Figures

Fig. 1
Fig. 1
Global Rating Scale. The GRS consists of ten items, each rated on a five-point Likert scale. The maximum overall score is 50 points, minimum score is 10 points
Fig. 2
Fig. 2
Task Specific Checklist. The Task Specific Checklist (TSC) complements the GRS, but is solely designed for non-traumatic patients. 25 items are either rated as ‘done correctly’ or ‘not done/incorrect’. CRT: capillary refill time, ecg: electrocardiogram, FAST: acronym meaning Face Arms Speech Time, SAMPLE: acronym meaning. Symptoms, Allergies, Medication, Past medical history, Last oral meal, Events
Fig. 3
Fig. 3
Graphical display of the overall results of the GRS. Median values of the overall scores of each group in the GRS are depicted. Median values of all four ratings per team that were used for further statistical analysis are also presented. P values are given for the Kruskal-Wallis-Test after Dunn-Bonferroni correction
See this image and copyright information in PMC

References

    1. Bond WF, Spillane L. The use of simulation for emergency medicine resident assessment. Acad Emerg Med. 2002;9(11):1295–1299. doi: 10.1197/aemj.9.11.1295. - DOI - PubMed
    1. Drews FA, Bakdash JZ. Simulation training in health care. Rev Hum Factor Ergonomics. 2013;8(1):191–234. doi: 10.1177/1557234X13492977. - DOI
    1. Beaubien JM, Baker DP. The use of simulation for training teamwork skills in health care: how low can you go? Qual Saf Health Care. 2004;13(Suppl 1):i51–i56. doi: 10.1136/qshc.2004.009845. - DOI - PMC - PubMed
    1. Issenberg SB. The scope of simulation-based healthcare education. Simul Healthc. 2006;1(4):203–208. doi: 10.1097/01.SIH.0000246607.36504.5a. - DOI - PubMed
    1. Issenberg SB, McGaghie WC, Hart IR, Mayer JW, Felner JM, Petrusa ER, et al. Simulation technology for health care professional skills training and assessment. JAMA. 1999;282(9):861–866. doi: 10.1001/jama.282.9.861. - DOI - PubMed

LinkOut - more resources