Computer-aided diagnosis system of thyroid nodules ultrasonography: Diagnostic performance difference between computer-aided diagnosis and 111 radiologists

Abstract

To evaluate the diagnostic efficiency of computer-aided diagnosis (CAD) system and 111 radiologists with different experience in identifying benign and malignant thyroid nodules, and to summarize the ultrasound features that may affect the diagnostic of CAD and radiologists.Fifty thyroid nodules and 111 radiologists were enrolled in this study. All the 50 nodules were diagnosed by the 111 radiologists and the CAD system simultaneously. The diagnostic performance of the CAD system, senior and junior radiologists with the maximum accuracy were calculated and compared. Interobserver agreement for different ultrasound characteristics between the CAD and senior radiologist were analyzed.CAD system showed a higher specificity than junior radiologist (87.5% vs 70.4%, P = .03), and a lower sensitivity than the senior radiologist and junior radiologist but the statistics were not significant (76.9% vs 86.9%, P > .5; 76.9% vs 82.6%, P > .5). The CAD system and senior radiologist got larger AUC than junior radiologist but the differences were not statistically significant (0.82 vs 0.76, respectively; P = .5). The interobserver agreement for the US characteristics between the CAD system and senior radiologist were: substantial agreement for hypoechoic and taller than wide (kappa value = 0.66, 0.78), and moderate agreement for irregular margin and micro-calcifications (kappa value = 0.52, 0.42).The CAD system achieved equal diagnostic accuracy to the senior radiologists and higher accuracy than the junior radiologists. The interobserver agreements in the US features between the CAD system and senior radiologist were substantial agreement for hypoechoic and taller than wide; moderate agreement for irregular margin and micro-calcifications. The location of a thyroid nodule and the feature of macrocalcification with wide acoustic shadow may influence the analysis of the CAD system.

Figure 1

Receiver operating characteristic (ROC) curves. Receiver operating characteristic curves for CAD system (AUC, 0.82), senior radiologist with maximum accuracy (AUC, 0.82), and junior radiologist with maximum accuracy (AUC, 0.76). AUC = area under the curve; CAD = computer-aided diagnosis.

Figure 2

The misdiagnosed thyroid nodules of radiologists. A. A small papillary carcinoma with colloid inside the lesion (arrow) was misdiagnosed as benign one by most radiologists (with an accuracy of 20.9%); B. A papillary carcinoma nodule (arrow) in Hashimoto thyroiditis was misdiagnosed as benign one by most radiologists (with an accuracy of 24.0%).

Figure 3

The misdiagnosed thyroid nodules of CAD system. A nodular goiter located near to the thyroid capsule and trachea was misdiagnosed as malignant nodule by CAD system. CAD system indicated that this nodule is markedly hypoechoic and heterogeneous nodule with the presence of echogenic foci and taller than wide shape. CAD = computer-aided diagnosis.