Quality control of radiomic features using 3D-printed CT phantoms

Abstract

Purpose: The lack of standardization in quantitative radiomic measures of tumors seen on computed tomography (CT) scans is generally recognized as an unresolved issue. To develop reliable clinical applications, radiomics must be robust across different CT scan modes, protocols, software, and systems. We demonstrate how custom-designed phantoms, imprinted with human-derived patterns, can provide a straightforward approach to validating longitudinally stable radiomic signature values in a clinical setting. Approach: Described herein is a prototype process to design an anatomically informed 3D-printed radiomic phantom. We used a multimaterial, ultra-high-resolution 3D printer with voxel printing capabilities. Multiple tissue regions of interest (ROIs), from four pancreas tumors, one lung tumor, and a liver background, were extracted from digital imaging and communication in medicine (DICOM) CT exam files and were merged together to develop a multipurpose, circular radiomic phantom (18 cm diameter and 4 cm width). The phantom was scanned 30 times using standard clinical CT protocols to test repeatability. Features that have been found to be prognostic for various diseases were then investigated for their repeatability and reproducibility across different CT scan modes. Results: The structural similarity index between the segment used from the patients' DICOM image and the phantom CT scan was 0.71. The coefficient variation for all assessed radiomic features was $<1.0\%$ across 30 repeat scans of the phantom. The percent deviation (pDV) from the baseline value, which was the mean feature value determined from repeat scans, increased with the application of the lung convolution kernel, changes to the voxel size, and increases in the image noise. Gray level co-occurrence features, contrast, dissimilarity, and entropy were particularly affected by different scan modes, presenting with $\mathrm{pDV}>\pm 15\%$ . Conclusions: Previously discovered prognostic and popular radiomic features are variable in practice and need to be interpreted with caution or excluded from clinical implementation. Voxel-based 3D printing can reproduce tissue morphology seen on CT exams. We believe that this is a flexible, yet practical, way to design custom phantoms to validate and compare radiomic metrics longitudinally, over time, and across systems.

Keywords: additive manufacturing; computed tomography; quantitative imaging; radiomics.

Fig. 1

Workflow to generate 3D-printed phantom. (a) The tumor was segmented from the patient CT exam. A cross-sectional slice from a single patient’s CT image shows the contoured PDAC. (b) Binary masks of the tumors were generated and then used to replace the background voxel values with the tumor voxel intensity values. (c) The combined volume was then supersampled to the resolution of the 3D printer and stacked into slices. Each slice from (c) was then dithered using the Floyd–Steinberg dithering algorithm into binary raster files. (d) Three sets of raster files were generated, one for each resin material. These files define the spatial location of each resin material. (e) Resultant 3D print of the combined volume. Due to the material used, visualizing the internal structure is not possible with the naked eye.

Fig. 2

Cross-sectional CT images of the radiomic phantom from each scan mode evaluated in this study. For each image, a window width of 40 and a level of 100 were applied. The differences between each scan mode are (a) baseline image reconstructed with standard kernel. (b) ASiR of 10%. (c) ASiR 20% and (d) ASiR 30%. (e) Lung kernel. (f) Lung kernel with ASiR 40%. (g) Bone kernel. (h) Reduced tube current of 100 mA. (i) Reduced tube potential of 100 kVp. (j) Dual-energy CT image reconstructed at monochromatic 70 keV. (k) Enlarged DFOV of 350 mm with a pixel size of 0.689 mm. (l) Phantom placed off-center by 30 mm. The off-center image was electronically centered within the field of view.

Fig. 3

(a) The 3D-printed radiomic phantom. (b) Axial slice generated from a CT scan shows the embedded tumors within the background tissue. (c) The tumor contours were generated, and the tumor types were labeled as 1–Non-small cell lung carcinoma; 2 to 5–Pancreatic ductal adenocarcinoma.

Fig. 4

(a) An axial slice from a patient CT showing the region of interest (ROI) around the heterogeneous hepatic tissue. (b) A cropped and expanded view of the portion of the patient’s liver on which the background of the 3D print was modeled. (c) An axial slice of the resulting 3D print CT scan.

Fig. 5

The within-subject coefficient of variation (wCV, %) for prognostic non-small cell lung carcinoma radiomic features extracted from each tumor. The wCV was computed from the 30 repeated CT scans acquired with the reference protocol. The 95th percentile confidence intervals are displayed for each feature value.

Fig. 6

The percent deviation (pDV, %) and one-sample Wilcoxon signed-rank test compare the prognostic NSCLC features. Comparisons are being made between the average feature value derived from the 30 repeat scans and the additional scan modes. (a) First-order energy, (b) gray- level non-uniformity (GLN), and HLH wavelet GLN.

Fig. 7

Dot plots of the change in radiomic feature values as a function of each scanning technique and tumor type.