Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2017 Apr 5:8:257-263.
doi: 10.2147/AMEP.S128963. eCollection 2017.

Development of performance and error metrics for ultrasound-guided axillary brachial plexus block

Osman M Ahmed  1 , Brian D O'Donnell  1   2 , Anthony G Gallagher  2 , George D Shorten  1   2
Affiliations

Development of performance and error metrics for ultrasound-guided axillary brachial plexus block

Osman M Ahmed et al. Adv Med Educ Pract. .

Abstract

Purpose: Change in the landscape of medical education coupled with a paradigm shift toward outcome-based training mandates the trainee to demonstrate specific predefined performance benchmarks in order to progress through training. A valid and reliable assessment tool is a prerequisite for this process. The objective of this study was to characterize ultrasound-guided axillary brachial plexus block to develop performance and error metrics and to verify face and content validity using a modified Delphi method.

Methods: A metric group (MG) was established, which comprised three expert regional anesthesiologists, an experimental psychologist and a trained facilitator. The MG deconstructed ultrasound-guided axillary brachial plexus block to identify and define performance and error metrics. Experts reviewed five video recordings of the procedure performed by anesthesiologists with different levels of expertise to aid task deconstruction. Subsequently, the MG subjected the metrics to "stress testing", a process to ascertain the extent to which the performance and error metrics could be scored objectively, either occurring or not occurring with a high degree of reliability. Ten experienced regional anesthesiologists used a modified Delphi method to reach consensus on the metrics.

Results: Fifty-four performance metrics, organized in six procedural phases and characterizing ultrasound-guided axillary brachial plexus block and 32 error metrics (nine categorized as critical) were identified and defined. Based on the Delphi panel consensus, one performance metric was modified, six deleted and three added.

Conclusion: In this study, we characterized ultrasound-guided axillary brachial plexus block to develop performance and error metrics as a prerequisite for outcome-based training and assessment. Delphi consensus verified face and content validity.

Keywords: assessment; metrics; task analysis; training; validation.

PubMed Disclaimer

Conflict of interest statement

Disclosure The authors report no conflicts of interest in this work.

Figures

Figure 1
Figure 1
A view from a video recording presented to the MG to aid procedure characterization. Views are from the operative procedure and ultrasound output. Abbreviation: MG, metric group.
See this image and copyright information in PMC

References

    1. Marhofer P, Greher M, Kapral S. Ultrasound guidance in regional anesthesia. Br J Anaesth. 2005;94(1):7–17. - PubMed
    1. Regional Anesthesiology and Acute Pain Medicine Fellowship Directors Group Guidelines for fellowship training in regional anesthesiology and acute pain medicine: second edition, 2010. Reg Anesth Pain Med. 2011;36(3):282–288. - PubMed
    1. Sites BD, Chan VW, Neal JM, et al. The American Society of Regional Anesthesia and Pain Medicine and the European Society of Regional Anaesthesia and Pain Therapy joint committee recommendations for education and training in ultrasound-guided regional anesthesia. Reg Anesth Pain Med. 2010;35(2 Suppl):S74–S80. - PubMed
    1. Bould MD, Crabtree NA, Naik VN. Assessment of procedural skills in anaesthesia. Br J Anaesth. 2009;103(4):472–483. - PubMed
    1. Epstein RM. Assessment in medical education. N Engl J Med. 2007;356(4):387–396. - PubMed

LinkOut - more resources