Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2020 Oct 3;5(3):e10520.
doi: 10.1002/aet2.10520. eCollection 2021 Jul.

The Ultrasound Competency Assessment Tool (UCAT): Development and Evaluation of a Novel Competency-based Assessment Tool for Point-of-care Ultrasound

Colin Bell  1 , Andrew K Hall  1 , Natalie Wagner  2   3 , Louise Rang  1 , Joseph Newbigging  1 , Conor McKaigney  4
Affiliations

The Ultrasound Competency Assessment Tool (UCAT): Development and Evaluation of a Novel Competency-based Assessment Tool for Point-of-care Ultrasound

Colin Bell et al. AEM Educ Train. .

Abstract

Objectives: Point-of-care ultrasound (POCUS) has become an integral diagnostic and interventional tool. Barriers to POCUS training persist, and it continues to remain heterogeneous across training programs. Structured POCUS assessment tools exist, but remain limited in their feasibility, acceptability, reliability, and validity; none of these tools are entrustment-based. The objective of this study was to derive a simple, entrustment-based POCUS competency assessment tool and pilot it in an assessment setting.

Methods: This study was composed of two phases. First, a three-step modified Delphi design surveyed 60 members of the Canadian Association of Emergency Physicians Emergency Ultrasound Committee (EUC) to derive the anchors for the tool. Subsequently, the derived ultrasound competency assessment tool (UCAT) was used to assess trainee (N = 37) performance on a simulated FAST examination. The intraclass correlation (ICC) for inter-rater reliability and Cronbach's alpha for internal consistency were calculated. A statistical analysis was performed to compare the UCAT to other competency surrogates.

Results: The three-round Delphi had 22, 26, and 26 responses from the EUC members. Consensus was reached, and anchors for the domains of preparation, image acquisition, image optimization, and clinical integration achieved approval rates between 92 and 96%. The UCAT pilot revealed excellent inter-rater reliability (with ICC values of 0.69-0.89; p < 0.01) and high internal consistency (α = 0.91). While UCAT scores were not impacted by level of training, they were significantly impacted by the number of previous POCUS studies completed.

Conclusions: We developed and successfully piloted the UCAT, an entrustment-based bedside POCUS competency assessment tool suitable for rapid deployment. The findings from this study indicate early validity evidence for the use of the UCAT as an assessment of trainee POCUS competence on FAST. The UCAT should be trialed in different populations performing several POCUS study types.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Ultrasound Competency Assessment Tool (UCAT).
Figure 2
Figure 2
(A) UCAT performance by level of training and number of FAST scans. (B) UCAT entrustment by level of training and number of FAST scans. (C) UCAT Performance by level of training and number of total scans. CCFP‐EM = Canadian College of Family Physicians Emergency Medicine; UCAT = Ultrasound Competency Assessment Tool.
See this image and copyright information in PMC

Similar articles

See all similar articles

Cited by

See all "Cited by" articles

References

    1. Solomon SD, Saldana F. Point‐of‐care ultrasound in medical education—stop listening and look. N Engl J Med 2014;370:1083–5. - PubMed
    1. Leschyna M, Hatam E, Britton S, et al. Current state of point‐of‐care ultrasound usage in Canadian emergency departments. Cureus 2019;11:e4246. - PMC - PubMed
    1. American College of Emergency Physicians . Ultrasound guidelines: emergency, point‐of‐care and clinical ultrasound guidelines in medicine. Ann Emerg Med 2017;69:e27–54. - PubMed
    1. Olszynski P, Kim DJ, Chenkin J, Rang L. The CAEP emergency ultrasound curriculum ‐ objectives and recommendations for implementation in postgraduate training: executive summary. CJEM 2018;20:736–8. - PubMed
    1. Blehar DJ, Barton B, Gaspari RJ. Learning curves in emergency ultrasound education. Acad Emerg Med 2015;22:574–82. - PubMed