Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
. 2022 Jan 1;11(1):e210023.
doi: 10.1530/ETJ-21-0023.

Reliability of a computer-aided system in the evaluation of indeterminate ultrasound images of thyroid nodules

J L Reverter  1   2 , L Ferrer-Estopiñan  1   2 , F Vázquez  1   2 , S Ballesta  1   2 , S Batule  1   2 , A Perez-Montes de Oca  1   2 , C Puig-Jové  1   2 , M Puig-Domingo  1   2
Affiliations

Reliability of a computer-aided system in the evaluation of indeterminate ultrasound images of thyroid nodules

J L Reverter et al. Eur Thyroid J. .

Abstract

Introduction: Computer-aided diagnostic (CAD) programs for malignancy risk stratification from ultrasound (US) imaging of thyroid nodules are being validated both experimentally and in real-world practice. However, they have not been tested for reliability in analyzing difficult or unclear images.

Methods: US images with indeterminate characteristics were evaluated by five observers with different experience in US examination and by a commercial CAD program. The nodules, on which the observers widely agreed, were considered concordant and, if there was little agreement, not concordant or difficult to assess. The diagnostic performance of the readers and the CAD program was calculated and compared in both groups of nodule images.

Results: In the group of concordant thyroid nodules (n = 37), the clinicians and the CAD system obtained similar levels of accuracy (77.0% vs 74.2%, respectively; P = 0.7) and no differences were found in sensitivity (SEN) (95.0% vs 87.5%, P = 0.2), specificity (SPE) (45.5 vs 49.4, respectively; P = 0.7), positive predictive value (PPV) (75.2% vs 77.7%, respectively; P = 0.8), nor negative predictive value (NPV) (85.6 vs 77.7, respectively; P = 0.3). When analyzing the non-concordant nodules (n = 43), the CAD system presented a decrease in accuracy of 4.2%, which was significantly lower than that observed by the experts (19.9%, P = 0.02).

Conclusions: Clinical observers are similar to the CAD system in the US assessment of the risk of thyroid nodules. However, the AI system for thyroid nodules AmCAD-UT® showed more reliability in the analysis of unclear or misleading images.

Keywords: CAD system; risk classification; thyroid nodules.

PubMed Disclaimer

Figures

Figure 1
Figure 1
Study design flowchart (see description in the text). SEN: sensitivity, SPE: specificity, PPV: positive predictive value, NPV: negative predictive value, ACC: accuracy.
Figure 2
Figure 2
Risk classification offered by the CAD system adequate for the pathological results in three undefined nodules from the study. The ultrasound image, delimitation and shape, echogenicity and hypochoic foci, margins and echogenic foci, and heterogeneity are presented in each nodule.
See this image and copyright information in PMC

References

    1. Durante C, Grani G, Lamartina L, Filetti S, Mandel SJ, Cooper DS. The diagnosis and management of thyroid nodules: a review. JAMA 2018319914–924. (10.1001/jama.2018.0898) - DOI - PubMed
    1. Burman KD, Wartofsky L. Thyroid nodules. New England Journal of Medicine 20153732347–2356. (10.1056/NEJMcp1415786) - DOI - PubMed
    1. Gharib H, Papini E, Garber JR, Duik DS, Harrell RM, Hegedüs L, Paschke R, Valcavi R, Vitti P. & AACE/ACE/AME Task Force on Thyroid Nodules. AACE/ACE/AME Task Force on thyroid nodules American Association of Clinical Endocrinologists, American College of Endocrinology, and Associazione Medici Endocrinologi Medical guidelines for clinical practice for the diagnosis and management of thyroid nodules–2016. Endocrine Practice 201622622–639. (10.4158/EP161208.GL) - DOI - PubMed
    1. Remonti LR, Kramer CK, Leitão CB, Pinto LC, Gross JL. Thyroid ultrasound features and risk of carcinoma: a systematic review and meta-analysis of observational studies. Thyroid 201525538–550. (10.1089/thy.2014.0353) - DOI - PMC - PubMed
    1. Ha EJ, Baek JH, Na DG. Risk stratification of thyroid nodules on ultrasonography: current status and perspectives. Thyroid 2017271463–1468. (10.1089/thy.2016.0654) - DOI - PubMed