Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2023 Jan 24;4(1):61-75.
doi: 10.34197/ats-scholar.2022-0063OC. eCollection 2023 Mar.

A Tool to Assess Competence in Critical Care Ultrasound Based on Entrustable Professional Activities

Hayley P Israel  1 , Martin Slade  2 , Katherine Gielissen  3 , Rachel B Liu  4 , Margaret A Pisani  5 , Astha Chichra  5
Affiliations

A Tool to Assess Competence in Critical Care Ultrasound Based on Entrustable Professional Activities

Hayley P Israel et al. ATS Sch. .

Abstract

Background: Existing assessment tools for competence in critical care ultrasound (CCUS) have limited scope and interrupt clinical workflow. The framework of entrustable professional activities (EPAs) is well suited to developing an assessment tool that is comprehensive and readily integrated into the intensive care unit (ICU) training environment.

Objective: This study sought to design an EPA-based tool to assess competence in CCUS for pulmonary and critical care fellows and to assess the validity and reliability of the tool.

Methods: Eight experts in CCUS met to define the core EPAs for CCUS. A nominal group technique was used to reach consensus. An assessment tool was created based on the EPAs with a modified Ottawa entrustability scale. Trained faculty evaluated pulmonary and critical care fellows using this tool in the ICU over a 6-month study period at a single institution. An assessment of validity of the EPA-based tool is made with four sources of validity evidence: content, response process, reliability, and relation to other variables. Reliability and response process data were generated using generalizability theory analysis to estimate sources of variance in entrustment scores. Analysis of response process validity and validity by relation to other variables was performed using regression models.

Results: Fifty-four assessments were recorded during the study period, conducted on 23 trainees by 13 faculty. Content validity of the tool was demonstrated using expert consensus and published guidelines from critical care societies to define the EPAs. Response process validity was demonstrated by the low variance in entrustment scores due to evaluators (0.086 or 6%) and high agreement between score and trainee self-assessment (regression coefficient, 0.82; P < 0.0001). Reliability was demonstrated by the high "true" variance in entrustment score attributable to the trainee: 0.674 or 45%. Validity by relation to other variables was demonstrated using regression analysis to show correlation between entrustment score and the number of times a fellow has performed an EPA (regression coefficient, 0.023; P < 0.0001).

Conclusion: An EPA-based assessment tool for competence in CCUS was created. We obtained sufficient validity evidence on three of the diagnostic EPAs. Procedural EPAs were infrequently assessed, limiting generalizability in this subgroup.

Keywords: assessment; critical care; intensive care unit; point-of-care ultrasound; validity.

PubMed Disclaimer

Comment in

  • doi: 10.34197/ats-scholar.2023-0006ED

Similar articles

See all similar articles

Cited by

References

    1. Díaz-Gómez JL, Mayo PH, Koenig SJ. Point-of-care ultrasonography. N Engl J Med . 2021;385:1593–1602. - PubMed
    1. Mayo PH, Beaulieu Y, Doelken P, Feller-Kopman D, Harrod C, Kaplan A, et al. American College of Chest Physicians/La Société de Réanimation de Langue Française statement on competence in critical care ultrasonography. Chest . 2009;135:1050–1060. - PubMed
    1. Howard ZD, Noble VE, Marill KA, Sajed D, Rodrigues M, Bertuzzi B, et al. Bedside ultrasound maximizes patient satisfaction. J Emerg Med . 2014;46:46–53. - PubMed
    1. Dodge KL, Lynch CA, Moore CL, Biroscak BJ, Evans LV. Use of ultrasound guidance improves central venous catheter insertion success rates among junior residents. J Ultrasound Med . 2012;31:1519–1526. - PubMed
    1. Razi R, Estrada JR, Doll J, Spencer KT. Bedside hand-carried ultrasound by internal medicine residents versus traditional clinical assessment for the identification of systolic dysfunction in patients admitted with decompensated heart failure. J Am Soc Echocardiogr . 2011;24:1319–1324. - PubMed