Auto-segmentation, radiomic reproducibility, and comparison of radiomics between manual and AI-derived segmentations for coronary arteries in cardiac [18F]NaF PET/CT images

Abstract

Background: [¹⁸F]NaF is a potential biomarker for assessing cardiac risk. Automated analysis of [¹⁸F]NaF positron emission tomography (PET) images, specifically through quantitative image analysis ("radiomics"), can potentially enhance diagnostic accuracy and personalised patient management. However, it is essential to evaluate the reproducibility and reliability of radiomic features to ensure their clinical applicability. This study aimed to (i) develop and evaluate an automated model for coronary artery segmentation using [¹⁸F]NaF PET and calcium scoring computed tomography (CSCT) images, (ii) assess inter- and intra-observer radiomic reproducibility from manual segmentations, and (iii) evaluate the radiomics reliability from AI-derived segmentations by comparison with manual segmentations.

Results: 141 patients from the "effects of Vitamin K and Colchicine on vascular calcification activity" (VikCoVac, ACTRN12616000024448) trial were included. 113 were used to train an auto-segmentation model using nnUNet on [¹⁸F]NaF PET and CSCT images. Reproducibility of inter- and intra-observer radiomics and reliability of radiomics from AI-derived segmentations was assessed using lower bound of intraclass correlation coefficient (ICC). The auto-segmentation model achieved an average Dice Similarity Coefficient of 0.61 ± 0.05, having no statistically significant difference compared to the intra-observer variability (p = 0.922). For the unfiltered images, 47(12.6%) CT and 25(7.5%) PET radiomics were inter-observer reproducible, while 133(35.8%) CT and 57(15.3%) PET radiomics were intra-observer reproducible. 7(9.7%) CT and 18(25.0%) PET first-order features, as well as 17(17.7%) CT GLCM features, were reproducible for both inter- and intra-observer analyses. 9.8% and 16.8% of radiomics from AI-derived segmentations showed excellent and good reliability. First-order features were most reliable (ICC > 0.75; 78/144[54.2%]) and shape features least (2/112[1.8%]). CT features demonstrated greater reliability (147/428[34.3%]) than PET (81/428 [18.9%]). Features from the left anterior descending (76/214[35.5%]) and right coronary artery (75/214[35.0%]) were more reliability than the circumflex (49/214[22.9%]) and left main (28/214[13.1%]) arteries.

Conclusions: An effective segmentation model for coronary arteries was developed and reproducible [¹⁸F]NaF PET/CSCT radiomics were identified through inter- and intra-observer assessments, supporting their clinical applicability. The reliability of radiomics from AI-derived segmentations compared to manual segmentations was highlighted. The novelty of [¹⁸F]NaF as a biomarker underscores its potential in providing unique insights into vascular calcification activity and cardiac risk assessment.

Clinical trial registration: VIKCOVAC trial ("effects of Vitamin K and Colchicine on vascular calcification activity"). Unique identifier: ACTRN12616000024448. URL: https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=368825 .

Keywords: Auto-segmentation; Coronary artery disease; Radiomics; Reproducibility; [18F]NaF PET.

Fig. 1

Comparison of auto-segmentation results with Intra-observer variability. Voxel-level distribution of DSC, precision, and recall metrics for individual coronary arteries, and averaged across all arteries for each of the 10 patients in the testing set with both secondary manual segmentations by the same observer and automated segmentations. P-values from Wilcoxon signed-rank tests are indicated for each metric comparison. The box plot shows the interquartile range (IQR) and whiskers for data within the 1.5 IQR

Fig. 2

Axial CSCT slices featuring manual and nnUNet segmentations for a patient. nnUNet segmentation metrics detailed as follows: Precision for LCx, LAD, LM, and RCA are 0.85, 0.83, 0.77, and 0.74, respectively; Recall for LCx, LAD, LM, and RCA are 0.68, 0.68, 0.43, and 0.68, respectively; DSC for LCx, LAD, LM, and RCA are 0.76, 0.75, 0.55, and 0.71, respectively. GT = ground truth manual segmentation, PRED = predicted automated segmentation

Fig. 3

Inter- and Intra- observer Radiomic Reproducibility. Proportion of features for different lower bound of ICC categories for the original unfiltered images across arteries, and imaging modalities (baseline CT, baseline [¹⁸F]NaF PET): (a) intra-observer radiomic reproducibility; (b) inter-observer radiomic reproducibility

Fig. 4

Reliability of AI-segmented features from original images. Bar chart illustrating the percentage of features from the original unfiltered CSCT images and [¹⁸F]NaF PET images using a 0.1 SUV bin width without mask dilation that have a lower bound of the ICC 95% CI above 0.75. The chart highlights the robustness of CT and PET features across different feature categories and coronary arteries, with the sequence of feature categories, coronary arteries, and image modalities arranged from left to right in descending order based on average percentage

Fig. 5

Figures illustrating the reliability of AI-segmented SUV_max feature from the original unfiltered [¹⁸F]NaF PET images. (a) Boxplot of SUV_max values for individual coronary arteries for each of the 10 patients in the testing set with both secondary manual segmentations by the same observer and automated segmentations. P-values from Wilcoxon signed-rank tests were calculated to compare SUV_max values between the automated segmentation and the first contour, the automated segmentation and the secondary contour, and between the two contours. The box plot shows the IQR and whiskers for data within the 1.5 IQR. (b) ICC values and their associated 95% CI plotted for SUV_max extracted from the original unfiltered [¹⁸F]NaF PET image. The ICC values are calculated for between the automated segmentation and the first contour, the automated segmentation and the secondary contour, and between the two contours

Fig. 6

Bar chart and heatmap illustrating the impact of mask dilation on AI-segmented feature reliability. (a) Lower bound of ICC value classification for radiomics features extracted from original, unfiltered [¹⁸F]NaF PET and CSCT images with no mask dilation and uniform 1 (DIL_1), 2 (DIL_2), 3 (DIL_3) voxel dilation. Features are categorised based on the lower end of the ICC 95% confidence interval using the Koo and Li classification scheme [26]; (b) Heatmap of ICC results depicting the proportion of features with excellent and good lower bound of ICC after applying uniform mask dilation of 1, 2, 3 voxels across imaging modalities (baseline CT, baseline PET), four coronary arteries, and feature categories, utilising a bin width of 0.1 SUV for PET images without any image filtering