Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Sep 1;38(9):442-447.
doi: 10.1097/PEC.0000000000002786. Epub 2022 Jun 24.

Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception

Kelly R Bergmann  1 , Marshal Khant  1 , Shea Lammers  1 , Alexander C Arroyo  2 , Pablo Avendano  3 , Lindsey Chaudoin  4 , Stephanie G Cohen  5 , J Kate Deanehan  6 , Aaron E Kornblith  7 , Samuel H F Lam  8 , Margaret Lin-Martore  7 , Laurie Malia  9 , Kathryn H Pade  10 , Daniel B Park  11 , Adam Sivitz  12 , Keren Shahar-Nissan  13 , Peter J Snelling  14 , Mark O Tessaro  15 , Rosemary Thomas-Mohtat  16 , Valerie Whitcomb  1 , Adriana Yock-Corrales  17 , Paige Walsh  18 , Dave Watson  18 , Manu Madhok  1 ; P2Network Intussusception Study Group
Affiliations

Accuracy and Interrater Reliability of Point-of-Care Ultrasonography Image Interpretation for Intussusception

Kelly R Bergmann et al. Pediatr Emerg Care. .

Abstract

Objectives: The aim of this study was to determine the accuracy and interrater reliability of (1) point-of-care ultrasound (POCUS) image interpretation for identification of intussusception and (2) reliability of secondary signs associated with intussusception among experts compared with novice POCUS reviewers.

Methods: We conducted a planned secondary analysis of a prospective, convenience sample of children aged 3 months to 6 years who were evaluated with POCUS for intussusception across 17 international pediatric emergency departments between October 2018 and December 2020. A random sample of 100 POCUS examinations was reviewed by novice and expert POCUS reviewers. The primary outcome was identification of the presence or absence of intussusception. Secondary outcomes included intussusception size and the presence of trapped free fluid or echogenic foci. Accuracy was summarized using sensitivity and specificity, which were estimated via generalized mixed effects logistic regression. Interrater reliability was summarized via Light's κ statistics with bootstrapped standard errors (SEs). Accuracy and reliability of expert and novice POCUS reviewers were compared.

Results: Eighteen expert and 16 novice POCUS reviewers completed the reviews. The average expert sensitivity was 94.5% (95% confidence interval [CI], 88.6-97.5), and the specificity was 94.3% (95% CI, 90.3-96.7), significantly higher than the average novice sensitivity of 84.7% (95% CI, 74.3-91.4) and specificity of 80.4% (95% CI, 72.4, 86.7). κ was significantly greater for expert (0.679, SE 0.039) compared with novice POCUS reviewers (0.424, SE 0.044; difference 0.256, SE 0.033). For our secondary outcome measure of intussusception size, κ was significantly greater for experts (0.661, SE 0.038) compared with novices (0.397, SE 0.041; difference 0.264, SE 0.029). Interrater reliability was weak for expert and minimal for novice reviewers regarding the detection of trapped free fluid and echogenic foci.

Conclusions: Expert POCUS reviewers demonstrate high accuracy and moderate interrater reliability when identifying intussusception via image interpretation and perform better than novice reviewers.

PubMed Disclaimer

Conflict of interest statement

Disclosure: The authors declare no conflict of interest.

References

    1. Sivitz AB, Cohen SG, Tejani C. Evaluation of acute appendicitis by pediatric emergency physician sonography. Ann Emerg Med . 2014;64:358–364.e4.
    1. Garcia AM, Asad I, Tessaro MO, et al. A multi-institutional case series with review of point-of-care ultrasound to diagnose malrotation and midgut volvulus in the pediatric emergency department. Pediatr Emerg Care . 2019;35:443–447.
    1. Malcom GE 3rd, Raio CC, Del Rios M, et al. Feasibility of emergency physician diagnosis of hypertrophic pyloric stenosis using point-of-care ultrasound: a multi-center case series. J Emerg Med . 2009;37:283–286.
    1. Riera A, Hsiao AL, Langhan ML, et al. Diagnosis of intussusception by physician novice sonographers in the emergency department. Ann Emerg Med . 2012;60:264–268.
    1. Bergmann KR, Arroyo AC, Tessaro MO, et al. Diagnostic accuracy of point-of-care ultrasound for intussusception: a multicenter, noninferiority study of paired diagnostic tests. Ann Emerg Med . 2021;78:606–615.

LinkOut - more resources