A deep learning model for carotid plaques detection based on CTA images: a two stepwise early-stage clinical validation study

Abstract

Objective: To develop a deep learning (DL) model for carotid plaque detection based on CTA images and evaluate the clinical application feasibility and value of the model.

Methods: We retrospectively collected data from patients with carotid atherosclerotic plaques who underwent continuous CTA examinations of the head and neck at a tertiary hospital from October 2020 to October 2022. The model combined ResUNet with the Pyramid Scene Parsing Network (PSPNet) to enhance plaque segmentation. Patient plaques were divided into training, validation, and testing sets in a ratio of 7:1.5:1.5. We analyzed recall (lesion-level sensitivity), sensitivity (patient-level), and precision to evaluate the model's diagnostic performance for carotid plaques. The two stepwise early-stage clinical validation study (Comparison study and Model-human study) was used to simulate real clinical plaque diagnostic scenarios.

Results: In total, 647 patients were included in the dataset, including 475 for training, 86 for validation, and 86 for testing. The DL model based on CTA images showed good precision in plaque diagnosis (validation set: precision = 80.49%, sensitivity = 90.70%, recall = 84.62%; test set: precision = 78.37%, sensitivity = 91.86%, recall = 84.58%). In addition, subgroup analysis of the plaque was carried out in the test set. The model had high accuracy in identifying plaques at different locations (Recall: 83.72, 76.32, 89.25, and 83.02%) and with different morphologies (Recall: 86.03, 79.17%). This model also analyzed the results of different types of plaques and showed good to moderate plaque diagnostic accuracy for different plaque types (Recall: 70.00, 86.87, 84.29%). Especially, in the clinical application scenario analysis, the model's diagnostic results for plaques were found to be higher than those of 4 out of 6 radiologists (p < 0.001). Furthermore, in Model-human Real Clinical Scenarios study, we found that the model improved the radiologists' sensitivity in diagnosing plaques. Additionally, the model's diagnostic time for plaques (6 s) was found to be significantly shorter than that all of radiologists (p < 0.001).

Conclusion: This AI model demonstrated strong clinical potential for carotid plaque detection with improved clinician diagnostic performance, shortening time, and practical implementation in real-world clinical cases.

Keywords: artificial intelligence; carotid plaque; computed tomography angiography; deep learning; head and neck.

Figure 1

The main structure of the work. It introduces the flow of the deep learning algorithm and the two-step clinical scenario validation of the algorithm model.

Figure 2

Partial presentation of modeling results. As shown in the Figure, the red border lines indicate the model’s recognition of the boundary of the plaques. (A,B) The models exhibited high recognition precision of calcified plaque and soft plaque on the left side of the neck. (C) The calcified plaque on the left side of the neck was not precisely identified, which may be due to its smaller size. (D) The plaque at the right carotid bifurcation was not completely and precisely identified, which may be attributed to the larger size and the location of partial plaque components near the edge.

Figure 3

Diagnostic results of comparison study. Among 6 radiologists, only D5 radiologists having higher plaque recall values than the model, indicating that the model has higher performance in plaque diagnosis and can assist physicians in improving the accuracy of plaque diagnosis. The model diagnosis time for plaques is 6 s, significantly shorter than the diagnosis time of 6 radiologists.

Figure 4

Diagnostic results of Model-human study. For plaque diagnosis, the results showed that 6 radiologists had improved diagnostic recall. The radiologists’ diagnoses (D7, D10 and D12), when aided by the model, exhibited a markedly higher degree of accuracy than model. Junior radiologist (D7) took longer to achieve a higher diagnostic performance and senior radiologists spent less time and achieved better performance. The model diagnosis time for plaques was 6 s, which was still significantly shorter than the diagnosis time of 6 radiologists.