Skip to main page content
U.S. flag

An official website of the United States government

Dot gov

The .gov means it’s official.
Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

Https

The site is secure.
The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
Meta-Analysis
. 2019 Aug;98(32):e16379.
doi: 10.1097/MD.0000000000016379.

Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis

Wan-Jun Zhao  1 , Lin-Ru Fu  2 , Zhi-Mian Huang  3 , Jing-Qiang Zhu  1 , Bu-Yun Ma  4
Affiliations
Meta-Analysis

Effectiveness evaluation of computer-aided diagnosis system for the diagnosis of thyroid nodules on ultrasound: A systematic review and meta-analysis

Wan-Jun Zhao et al. Medicine (Baltimore). 2019 Aug.

Abstract

Background: More and more automated efficient ultrasound image analysis techniques, such as ultrasound-based computer-aided diagnosis system (CAD), were developed to obtain accurate, reproducible, and more objective diagnosis results for thyroid nodules. So far, whether the diagnostic performance of existing CAD systems can reach the diagnostic level of experienced radiologists is still controversial. The aim of the meta-analysis was to evaluate the accuracy of CAD for thyroid nodules' diagnosis by reviewing current literatures and summarizing the research status.

Methods: A detailed literature search on PubMed, Embase, and Cochrane Libraries for articles published until December 2018 was carried out. The diagnostic performances of CAD systems vs radiologist were evaluated by meta-analysis. We determined the sensitivity and the specificity across studies, calculated positive and negative likelihood ratios and constructed summary receiver-operating characteristic (SROC) curves. Meta-analysis of studies was performed using a mixed-effect, hierarchical logistic regression model.

Results: Five studies with 536 patients and 723 thyroid nodules were included in this meta-analysis. The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio (DOR) for CAD system were 0.87 (95% confidence interval [CI], 0.73-0.94), 0.79 (95% CI 0.63-0.89), 4.1 (95% CI 2.5-6.9), 0.17 (95% CI 0.09-0.32), and 25 (95% CI 15-42), respectively. The SROC curve indicated that the area under the curve was 0.90 (95% CI 0.87-0.92). The pooled sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and DOR for experienced radiologists were 0.82 (95% CI 0.69-0.91), 0.83 (95% CI 0.76-0.89), 4.9 (95% CI 3.4-7.0), 0.22 (95% CI 0.12-0.38), and 23 (95% CI 11-46), respectively. The SROC curve indicated that the area under the curve was 0.96 (95% CI 0.94-0.97).

Conclusion: The sensitivity of the CAD system in the diagnosis of thyroid nodules was similar to that of experienced radiologists. However, the CAD system had lower specificity and DOR than experienced radiologists. The CAD system may play the potential role as a decision-making assistant alongside radiologists in the thyroid nodules' diagnosis. Future technical improvements would be helpful to increase the accuracy as well as diagnostic efficiency.

PubMed Disclaimer

Conflict of interest statement

The authors have no conflicts of interest to disclose.

Figures

Figure 1
Figure 1
Flow chart of the search for eligible studies on diagnostic performance of computer-aided diagnosis system.
Figure 2
Figure 2
The results of methodologic quality analysis for each study in meta-analysis according to Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2).
Figure 3
Figure 3
Summary of results of methodologic quality analysis of 5 studies in meta-analysis according to Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2).
Figure 4
Figure 4
Results of Deeks’ funnel plot of asymmetry test for publication bias. Nonsignificant slope indicates that no significant bias was found.
Figure 5
Figure 5
Forest plot of pooled sensitivity and specificity of computer-aided diagnosis system for the diagnosis of thyroid nodules.
Figure 6
Figure 6
Forest plot of diagnostic odds ratio of computer-aided diagnosis system for the diagnosis of thyroid nodules.
Figure 7
Figure 7
Summary receiver-operating characteristic (SROC) curves of computer-aided diagnosis system for the diagnosis of thyroid nodules.
Figure 8
Figure 8
Forest plot of pooled sensitivity and specificity of experienced radiologists for the diagnosis of thyroid nodules.
Figure 9
Figure 9
Forest plot of diagnostic odds ratio of experienced radiologists for the diagnosis of thyroid nodules.
Figure 10
Figure 10
Summary receiver-operating characteristic (SROC) curves of experienced radiologists for the diagnosis of thyroid nodules.
Figure 11
Figure 11
Forest plot of pooled sensitivity and specificity of computer-aided diagnosis system by Samsung for the diagnosis of thyroid nodules.
Figure 12
Figure 12
Forest plot of diagnostic odds ratio of computer-aided diagnosis system by Samsung for the diagnosis of thyroid nodules.
Figure 13
Figure 13
Summary receiver-operating characteristic (SROC) curves of computer-aided diagnosis system by Samsung for the diagnosis of thyroid nodules.
Figure 14
Figure 14
The concrete implementation flow of most existing computer-aided diagnosis systems based on neural network model.
See this image and copyright information in PMC

References

    1. Guth S, Theune U, Aberle J, et al. Very high prevalence of thyroid nodules detected by high frequency (13 MHz) ultrasound examination. Eur J Clin Invest 2010;39:699–706. - PubMed
    1. Wolinski K, Stangierski A, Ruchala M. Comparison of diagnostic yield of core-needle and fine-needle aspiration biopsies of thyroid lesions: systematic review and meta-analysis. Eur Radiol 2017;27:431–6. - PMC - PubMed
    1. Siegel RL, Miller KD, Jemal A. Cancer statistics, 2018. CA Cancer J Clin 2018;68:7–30. - PubMed
    1. Xiaoyi L, General Do. The interpretation of 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: surgery part. China Oncol 2017;52:309–15. - PubMed
    1. Haugen BR, Alexander EK, Bible KC, et al. 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 2016;26:1–33. - PMC - PubMed