Lung tumor segmentation methods: Impact on the uncertainty of radiomics features for non-small cell lung cancer

Abstract

Purpose: To evaluate the uncertainty of radiomics features from contrast-enhanced breath-hold helical CT scans of non-small cell lung cancer for both manual and semi-automatic segmentation due to intra-observer, inter-observer, and inter-software reliability.

Methods: Three radiation oncologists manually delineated lung tumors twice from 10 CT scans using two software tools (3D-Slicer and MIM Maestro). Additionally, three observers without formal clinical training were instructed to use two semi-automatic segmentation tools, Lesion Sizing Toolkit (LSTK) and GrowCut, to delineate the same tumor volumes. The accuracy of the semi-automatic contours was assessed by comparison with physician manual contours using Dice similarity coefficients and Hausdorff distances. Eighty-three radiomics features were calculated for each delineated tumor contour. Informative features were identified based on their dynamic range and correlation to other features. Feature reliability was then evaluated using intra-class correlation coefficients (ICC). Feature range was used to evaluate the uncertainty of the segmentation methods.

Results: From the initial set of 83 features, 40 radiomics features were found to be informative, and these 40 features were used in the subsequent analyses. For both intra-observer and inter-observer reliability, LSTK had higher reliability than GrowCut and the two manual segmentation tools. All observers achieved consistently high ICC values when using LSTK, but the ICC value varied greatly for each observer when using GrowCut and the manual segmentation tools. For inter-software reliability, features were not reproducible across the software tools for either manual or semi-automatic segmentation methods. Additionally, no feature category was found to be more reproducible than another feature category. Feature ranges of LSTK contours were smaller than those of manual contours for all features.

Conclusion: Radiomics features extracted from LSTK contours were highly reliable across and among observers. With semi-automatic segmentation tools, observers without formal clinical training were comparable to physicians in evaluating tumor segmentation.

Fig 1. Tumor presentations and locations.

A central slice of each tumor in the coronal view is displayed to show the variety in tumor locations, shapes and appearances of the patients used in this study. A single physician contour is displayed (red) to identify the tumor in each patient scan.

Fig 2. Schematic of the collection of manual and semi-automatic contours.

Each circle and triangle represent a single tumor contour. The time interval between contour set 1 and contour set 2 was 1 year for the contours represented by circles and 1 month for the contours represented by triangles.

Fig 3. User inputs for initializing semi-automatic segmentation tools.

(A) LSTK requires the user to select a seed within the tumor (red) to initiate the segmentation algorithm. Defining the maximum tumor radius generates a 3D bounding box (green) centered about the seed, within which the segmentation result will be confined. (B) GrowCut requires the user to label foreground (blue) and background (yellow) pixels to initiate the segmentation algorithm. Once labels were established, the GrowCut algorithm was followed by manual editing of the GrowCut-generated contours. Note that only the transverse view is shown here. Observers also labeled foreground and background pixels in the coronal and sagittal planes for each tumor case.

Fig 4. Validating segmentation accuracy of semi-automatic contours.

Box plot of the Dice similarity coefficients and Hausdorff distances by software tool displays the segmentation accuracy for each software tool.

Fig 5. Histogram distribution of the normalized dynamic range for all 83 radiomics features.

The histogram distribution shows the number of features within a range of NDR values where each bin has a width of 0.05.

Fig 6. Spearman correlation coefficient heat map including all initial 83 features.

Spearman correlation coefficients were computed for 83 radiomics features. Green, white, and red denote positive, random, and negative correlations, respectively. A large number of features were highly correlated.

Fig 7. Spearman correlation coefficient heat map including 40 non-redundant features.

Feature pairs with Spearman correlation coefficients less than 0.95. Spearman correlation coefficients larger than 0.95 were regarded as highly redundant and were eliminated from the initial feature set, reducing the feature set to 40 non-redundant features. Green, white, and red denote positive, random, and negative correlations, respectively. Correlation coefficients marked with an x are insignificant coefficients.

Fig 8. Intra-observer reliability.

Box plot of ICCs for each intra-observer relationship. ICC values were computed between contour run 1 and contour run 2 for each feature. Each physician/observer and software tool combination is plotted along the x-axis. Intra-observer reliability was observer-dependent. All observers achieved excellent feature reliability with LSTK.

Fig 9. Inter-observer reliability.

Box plot of ICCs for each inter-observer relationship. The ICC values were computed between all physician/observer contours for each feature. Each contour run and software tool combination is plotted along the x-axis. Inter-observer reliability was superior with LSTK compared with all other software tools.

Fig 10. Inter-software reliability.

Box plot of ICCs for each inter-software relationship. The ICC values were computed between contours generated by two different software tools for each feature. Each contour run and segmentation method combination is plotted along the x-axis. Inter-observer reliability was relatively low for all inter-software relationships, with the ICC values for many features falling within the poor classification.

Fig 11. ICC classification of each radiomics feature for each ICC relationship.

Red, orange, yellow, and green cells denote the ICC classifications of poor (ICC < 0.4), fair (0.4 ≤ ICC < 0.60), good (0.60 ≤ ICC < 0.75), and excellent (0.75 ≤ ICC) reproducibility, respectively [33].

Fig 12. Wilcoxon rank sum results between intraclass correlation coefficients for different feature categories.

Asterisks indicate that the median ICC was significantly different (p<0.05) between the two feature categories being compared. Blue cells indicate that the reproducibility of texture features was significantly less than the reproducibility of shape features. Red cells indicate that the reproducibility of texture features was significantly greater than the reproducibility of shape features.

Fig 13. Normalized feature range.

Comparison of normalized feature range between manual and semi-automatic methods using z-score normalization. The minimum and maximum values are plotted for each feature and segmentation method.